Latest / Agentic Conversations (formally mlops.community) / Just Fetch the Data and then... // David Bayliss // Coffee Sessions #110

Transcript

Just Fetch the Data and then... // David Bayliss // Coffee Sessions #110 · Agentic Conversations (formally mlops.community)

0:00
LexisNexis is probably the coolest name because it Rhymes
0:05
Lexis Nexis. You could just say that a few
0:08
times, it's like my buddy. He's trying to think of a name
0:13
for a start-up right now, and I just wasn't that day to double
0:19
check is a good name, right? It is.
0:26
That's the whole point of the company.
0:30
Honestly, I got it right from the name that makes sense.
0:33
They to double check. I think there's new Consulting
0:36
business consultant business for you now.
0:38
Yes, I naming dude, should I go? And, you know, they have start
0:42
up naming like Consultants. I have heard yeah.
0:45
Like branding and stuff, but I just think you can just simplify
0:48
it. Yeah.
0:49
Just this is your space to disrupt.
0:51
Yeah. And I figure out if the URL is,
0:54
or if like the domain is free and yeah, let's go for it.
1:00
I'll give you all that kind of stuff.
1:02
Alright, anyways today we talked to David Bayless from LexisNexis
1:06
risk, dude. Would you think it was an
1:09
awesome conversation? David is kind of like an unknown
1:15
Legend to me because for those who aren't familiar and you
1:18
probably aren't he is the chief data scientist at LexisNexis,
1:22
which is a company that definitely has your information
1:25
and is using it to power. All sorts of use cases like how
1:28
you get your home loan, how somebody does a background check
1:31
on you, how somebody figures out, you know, you're dressed
1:35
for validation, for whatever it might be your bank or anyone
1:38
else. And he has been there for 20
1:40
years and started off doing very low level computer.
1:44
I mean work and has used that now to help make the world of
1:49
large-scale data science for individual verification a lot
1:52
better. What was your main takeaway?
1:54
Well, they definitely have your information but in a good way
1:57
like I would I want these guys out the information after
2:02
hearing what David was talking about and how he thinks through
2:07
these really difficult problems and I thought he's awesome to
2:12
listen to, he's just funny in In the way that he doesn't try to
2:15
be funny. And then he just has these
2:18
incredible moments where you realize he did not try and crack
2:21
a joke or anything, but that was actually quite quite funny.
2:25
The way he said that, and he originally British this
2:29
appointment. Yeah, exactly.
2:31
Like I appreciated that piece of it and it's that a British dry
2:36
humor to. So but comes back to like the
2:39
content that we were talking about.
2:41
I found David just off. With him from some LinkedIn
2:46
posts and it was very random. I noticed a LinkedIn post where
2:50
he dug into some of the stuff they're doing and we get into
2:54
that in the conversation. But really like when I saw this
2:59
LinkedIn post, I realized that it was not your run-of-the-mill
3:03
like intro to ml Ops post and he talks about this low-level
3:09
stuff. He also talks about why he's got
3:11
phds that are doing so much. The low-level stuff and then he
3:15
doesn't even want the data scientists or the ml engineer's.
3:19
Even to be messing with any of that low level stuff and they're
3:22
throwing. What do you say like a dozen or
3:24
more phds just to figure out how to make things run, faster and
3:28
smoother and more securely. And when I followed up with him,
3:32
after the talk, it was funny because I was like, what should
3:36
we like, name the talk that we had, like, what should we have
3:40
the title of this be and he was like, hmmm.
3:44
Like data governance, security and privacy.
3:47
It's not really the sexiest, quick key, click Beatty thing
3:51
that we could do and I just thought that was like a
3:54
testament to how funny he is. Like he doesn't even try to be,
3:58
but he is, and so, yeah, that's it.
4:01
Like, I encourage everyone to check this out, like, listen to
4:05
this one. This is one that I feel like I'm
4:07
going to go back and listen to a few times, just because there
4:09
was so many gems and so many things, he said that 20 years of
4:14
Deep deep, focus in the field. It comes out here.
4:18
I totally agree. So with that, let's get into it.
4:23
And before we do, I want to mention that if anyone has any
4:27
recommendations on how we can make this show better, please
4:31
reach out to us. We're always looking to know
4:34
what we can do to make this more engaging more interesting for
4:38
you. The listener and it would mean a
4:41
ton to both Vishnu and I If you share this episode with one
4:47
friend that you think might be interested in what David has to
4:50
say. Alright, let's jump in.
4:56
Let's start with LexisNexis and your role in LexisNexis what
5:00
you're doing there, okay? LexisNexis is really about risk
5:06
analytics, whenever someone's taking a risk, they want them
5:10
most and best information they can to make the best informed
5:14
Decision, they can and that's where we come in.
5:17
So you'll see this in financial markets.
5:21
When you take out a bank account you probably won't to us.
5:23
If you ask for credit it will be went through us.
5:26
When you login and your app wants to know, is that really
5:29
you on the end of your phone, that probably went to us when
5:33
Financial transactions, go between Banks and they're trying
5:36
to make sure they're not doing money laundering, that for will
5:39
be went through Us in the insurance Market.
5:42
To, if you read, Reach out for an online quote, that probably
5:46
went through us when someone hires a school teacher at your
5:52
school. And they want to make sure this
5:53
isn't a sexpert that's hiding that probably went to us.
5:58
When you go to your favorite TV show and they said oh I just
6:02
want to find out who's in this area and what they've done and
6:04
what their background is that will have gone through us when
6:09
they're checking whether or not your kid is the actual kid.
6:12
Taking the exam that Will be went through us.
6:16
So really we are about 40 milliseconds of everything.
6:23
You do that matters and that's really the key thing that I I'm
6:30
hoping to get out of today. There's lots of cool stuff and
6:34
fun stuff out in the ml space in the AI space but my job is to be
6:40
the most boring person on the planet and to get the answer
6:43
right when it Really matters and I'm the chief data scientist for
6:48
LexisNexis, they didn't know what to call me 20 years ago
6:53
because they didn't have the term data science.
6:56
So I've been the data bunny, the data Alchemist him, that does
7:01
stuff or you. I mean anything it takes but my
7:06
job has always been, how do you actually take the data and
7:10
integrate it so that you can deploy it against the problem?
7:15
So many questions I have there and I think one is you've been
7:21
with LexisNexis for 20 years. I want to hear about the
7:24
evolution of those 20 years and what you've seen change since
7:30
you've been in data that whole time, but also like there's so
7:34
many use cases and it feels like that.
7:38
How do you, how do you even deal with all these different use
7:41
cases? Is it templated under the Or I
7:46
feel like you, you can't template all of this.
7:49
These use cases are so extremely different and so maybe we can
7:54
start with that and then go into the evolution of LexisNexis and
7:57
especially your evolution in the data field.
8:00
Well, I actually have a painfully simple answer to how
8:04
do you deal with all the use cases and that's you just ignore
8:07
them. They don't really matter because
8:10
there are so many use cases and they grow so far.
8:14
And they have evolved so much over time.
8:17
But the one thing you learn is that the use case that the
8:20
product manager thinks he has today is completely irrelevant
8:23
to what is eventually going to want my system to do, what
8:27
doesn't change, is the data that's coming in.
8:30
So my Approach is entirely focused upon the data.
8:34
Collecting the data integrate the data, what is the data
8:39
actually telling me what is the truth?
8:43
And then once I've built up my Well, the model they can ask
8:47
whatever questions they want to meet in, just answer them from
8:50
the model. I really, I mean I got to say I
8:53
love I love that perspective of saying well, but the saying to
8:57
me, now isn't what they're going to say to me then and I have to
9:00
spoke us or then meaning in the future, right?
9:03
I have to focus on what will always be around, which is the
9:08
need for high-quality data, understanding the data and
9:11
making sure that we have rational ways of organizing
9:14
excited Using a quality checking that data and just to provide
9:18
the lesson errs, those a little bit of a bit of context about
9:21
scalar data, that we're talking here looking at the company's
9:24
websites. That the company's statistics
9:27
that we pulled earlier here, LexisNexis has 1.9 million new
9:32
legal documents that are added daily, 71,000 sources,
9:36
generating 137 billion. Connections next, this news
9:41
includes over 39,000 premium sources.
9:44
This content includes more than two hundred, seventy three
9:47
million court, dockets over 140, million patent documents, I
9:50
could go on and on and on here, but just to kind of give the
9:54
listeners a sense of this is an incredible amount of information
9:58
that has to be organized and turned into a useful, almost
10:03
graph that can support a number of different use cases.
10:07
And my question to you, David here is how as the chief data
10:14
scientist do. play a role in the way that the company thinks
10:17
about Data and structuring data, as the tools for working with
10:22
data change, when we kind of have it split.
10:26
I like to think of it as a healthy schizophrenia in that my
10:30
job as the chief data scientist is the make sure that the data
10:35
has Integrity. So you spoke to part of our
10:38
scale, but that's in fact, the smaller part of the company, the
10:41
bigger part of the company has a quarter of a million different
10:44
sources and we're bringing in billions of records every day,
10:47
1.2 petabytes. Consolidated weekly.
10:51
It's you it's absolutely huge. My job is to make sure that we
11:00
have all of that data organized in a way that the data
11:04
scientists can get at. We have literally hundreds of
11:07
data scientists that are specialized out into all the
11:11
different verticals. So you know way way back in the
11:15
day 20 years ago I actually was what I would call a real data
11:19
scientist. I would go and talk to the
11:20
customers. I would find out what their
11:22
problems were and I was actually developing solutions for a real
11:27
customer But as time has progressed and moving like 150
11:32
countries and, you know, we have people now working in research,
11:37
departments in medical Fields, there's no way I can understand
11:40
PhD level discussions in 30 different fields.
11:44
So we employ the actual data scientist who go out and solve
11:48
the actual customer problems. But my job in the background is
11:53
to make sure that the data is integrated, but the other thing
11:57
we haven't touched upon is regulation in that.
12:01
If you imagine the data, we have is incredibly sensitive.
12:05
It is ridiculously sensitive. We know way more about people
12:09
than you would ever want to be publicly known and Therefore, we
12:13
operate under a very large number of regulations.
12:16
And a lot of my job is, actually making sure not that we have all
12:21
of the data about you to answer the different problem but making
12:24
sure we only have that data. But we can legally make
12:28
available available in that situation and that can differ by
12:33
customer if you given one set of permissions to One bank and
12:37
another set of permissions to another bank and then you apply
12:39
to both banks for a loan. They actually To get different
12:43
answers back. That's the law and making sure
12:47
that that Complex, dance happens, correctly is where I
12:53
come in and specifically we work at such scale that the little
12:58
tools you see out there literally just fall over and
13:01
die. You know, one of our main
13:04
feature extraction routines, it creates 120,000 features based
13:09
upon logically joining 720 files together.
13:14
Will you go up to your favorite SQL system and give it a select
13:17
statement with Doings. You will probably actually crash
13:21
the SQL Server. You certainly won't get an
13:24
answer back in 30 milliseconds, so my team has actually built
13:29
the tools to allow us to do that.
13:32
Work at the scale we need and I'm hoping we'll come on to
13:37
later also with the longevity that we need in that, we're not
13:41
a little startup, that's here to make a quick bang.
13:43
And then, you know, get out fast before the excrement, hits the
13:48
fan. We've been around for decades
13:51
and we plan to be around for decades and so our Solutions
13:54
have to be able to be resilient over decades.
13:59
Well, first off, I just want to compliment you on making the
14:03
same shit. Hits, the fan sound so much
14:06
better when you use excrement. I think that's great, second of
14:11
all. There's so many places that I
14:13
want to go and so many things that I would love to talk about
14:16
with this because it must be Guessing that you're doing
14:19
everything on premise your because of the I just feel like
14:26
if you were doing it in the cloud it would be really really
14:29
expensive but you can correct me on that.
14:32
And then the other thing is that I want to touch on is data
14:35
management and just data access with the ability like how do you
14:40
let these? How do you know which data
14:42
scientists have access to what, and how do you control that?
14:46
Have you built internal tools for the day?
14:48
Data access also yes. Okay.
14:52
So tackling first, we all come and lie on premise.
14:55
We actually built a supercomputer system a lot like
14:59
Hadoop but almost a decade before Hadoop and we have been
15:03
running their own premise for 20 years now.
15:07
And for many, many years we believed we had to stay on
15:11
purpose because it wasn't really feasible up in the cloud.
15:14
I mean yeah, a big batch process taking the one-and-a-half
15:17
petabytes of data speed. Now, 150 terabytes of confessed
15:21
Keys. That's like an eighty thousand
15:23
dollar process, and then you figured out, you've got a bug on
15:26
the third line and Boots. Yeah, I agree.
15:29
Now, in fact, I have come up with a plan to solve that so we
15:33
are migrating into the cloud but data access.
15:38
So the cord there is metadata now metadata is used way too
15:44
much so let me explain what it means.
15:47
We can't afford black. Boxes, most of our staff.
15:51
As I say, we making decisions are going to affect your life
15:54
and if you don't like the decision we made, you're going
15:56
to call up someone and whine about it.
15:59
And we have to be able to defend that decision.
16:02
So just about everything we do is highly explainable.
16:08
So our algorithms are effectively encoded in a DSL in
16:13
a metal language so that then all the way from the model we
16:17
can trace. The way through to the data,
16:20
what data was used White made those decisions.
16:23
Many of our algorithms actually have to be registered with a
16:26
state or with the customer, so that then when their customer
16:30
calls them, they can defend it and say, well no, I'm sorry, but
16:34
you got this conviction on this date and that disqualifies you
16:37
from this world. Now, that's a metadata, that
16:41
allows us to track a query that's happening in production
16:45
also allows us to track the data that's going to an individual
16:48
data. Scientist in the issue of
16:50
Imagine when a data scientist is doing something was we all love
16:55
to pretend it's blue cloud and blue sky?
16:57
And what-have-you the fact is they going in on a mission that
17:00
they could been told they need to discover something for some
17:02
customer and based upon what they have been told they need to
17:06
discover, we're going to know what kind of data they're going
17:10
to be allowed access to. And that is therefore, in fact,
17:14
filter through the algorithms. So if you imagine you Load up
17:19
with your algorithm and you actually provide a permission
17:23
block that comes from our security systems that say these
17:28
are the security rights you have when you're accessing this data
17:32
and the data that action comes up to you has then been filtered
17:35
down to that which you're allowed to have in that
17:37
situation. So make sense, it does, make
17:42
sense, it does make sense. And I think What I really
17:48
appreciate their is how you've laid out in the last for a few
17:51
minutes here at a high level, what kinds of meat challenges of
17:55
scale use case and I think diversity that, you know,
18:00
LexisNexis deals with right. You're working with Incredible
18:03
amount of data. You're working with highly
18:04
sensitive use cases and you're working across a very diverse
18:07
range and what I would really like our audience tends to be, I
18:12
would say practitioners that work in smaller teams.
18:17
Of eighties, right? Usually, like, for example, I
18:21
have primarily worked in teams of one, two, three.
18:23
Many of our listeners, tend to be, I would say up, two teams of
18:27
about 20, you know, a lot of people working in startups grow
18:30
stage companies, and even the public companies that, you know,
18:34
our listeners tend to be part of the data teams number, no, more
18:36
than 20, 30, maybe even 50, right?
18:40
And I think you're talking about sort of level of sensitivity and
18:43
frankly, maturity given the fact that you've been a part of these
18:46
efforts for 20 years, there's that many of them can certainly
18:48
learn from but maybe not relate to.
18:51
And so what I would love to talk through is the perspective of
18:54
like a single use case or a single data scientist, kind of
18:57
walk through the flow that they might have from being assigned
19:03
to a use case and ultimately deploying evaluating and
19:07
monitoring monitoring monitoring an algorithm for Success.
19:12
How does that sound? Yeah, I'll have to be vague in a
19:16
few places because we might think we have a bit of Secret
19:19
Sauce in there but sir, totally totally respect that.
19:24
Let me think through I'll go with one of the very I hope easy
19:29
to relate to cases. I talked about the fact that we
19:33
work in the insurance base. If you think about it, the idea
19:38
of insurance is you come along and you think you drive badly,
19:42
but you think you can persuade the K company, you drive well
19:46
and therefore it actually makes good Financial sense for you to
19:50
pay $300 a month. So that when you write off,
19:54
you'll be more once a year, they pay the bill rather than you.
19:59
There is a sort of vanilla in solvents algorithm that can give
20:03
a generalized waiting for you. But if all of the insurance
20:09
companies use that same vanilla algorithm, where's their value,
20:14
add how do they make money? What in fact, they're going to
20:18
sit around and do is they're going to decide there's some
20:21
part of the market but sweet. We're at the moment, the other
20:25
insurance companies are building that too high and they can take
20:29
a In more aggressive play and therefore that's what they want
20:34
to do. But obviously they want to know
20:36
how much I was going to cost them and they want to know just
20:39
how wide they can open the net for what degree of risk.
20:41
So typically they going to want to custom model.
20:44
So they will come from marketing department or the bigger
20:50
companies, will have a individual sales person attached
20:54
to them, and they will say we've got this analytics opportunity,
20:58
we want you to provide Us a better model for cost coverage
21:02
for this. And particular data scientist
21:07
will be assigned and they will be told initially the customer
21:12
is willing to give you the following amount of data with
21:15
outcomes. Now the particularly interesting
21:19
thing there is that you will actually get different outcomes
21:22
by company because the way as I feel important point the way
21:26
your model is handled also impacts the outcome of the
21:29
model. So for example if you have a set
21:34
of people who respond to an insurance claim by just telling
21:37
everyone to snap off and two-thirds of the people
21:41
actually take that off. Then you're going to actually
21:44
have a different cost response to that model.
21:46
Then some company that goes out of their way to say yes to
21:50
everybody automatically White's out the check.
21:52
So the moment you actually get specialist training data, that
21:57
changes it. So this data scientist will now
22:00
have that customer data, they will put that customer data for
22:04
our normal cleaning, a linking processes and they will pick a
22:07
date because remember, we're doing predictive models.
22:10
So they want to be able to prove to the customer that this would
22:15
have worked. So part of of our magic is our
22:17
system can't just produce all the data with the day.
22:20
It can produce the data as of any point in time over the last
22:23
10 years. This is how this thing, got
22:26
nicknamed, the Tardis we can travel through time so we wind
22:29
back a year ago and we produce the data dump that we would have
22:34
produced for these people a year ago and they now do that with
22:39
their outcome data. So they can say, this is what we
22:42
have had, this is what we're trying to predict.
22:45
Now at that point, if you might do we go to Balancing Act in the
22:49
on my side. And this is why we have, what I
22:51
call a healthy schizophrenia, I have to be so locked down and
22:55
uptight. I squeak when I walk but you
22:58
want the data scientists to be creative and you want them to be
23:02
open. So we don't typically restrict
23:05
what tools they use, but the ridden iron box, they can't get
23:12
any data out of that box. They can do what they want.
23:15
That they can produce some algorithm but they're in an iron
23:18
box. It stays there.
23:21
Then once they think they've got something marvelous and they
23:24
normally do, they have to code that up in my metalanguage?
23:28
Once it's in my metalanguage, I know exactly what it does.
23:32
And this thing can be, then be picked up and moved back to my
23:35
side of the world, where it can be executed in the production
23:38
environment and we know exactly what data it takes into account.
23:42
We can absolutely guarantee. It's not penalizing any of the
23:45
groups. We're not allowed to penalize
23:48
and all of that kind of good stuff.
23:50
So that is a typical cycle. Now, I can't really tell you
23:53
exactly what they're going to do.
23:55
Some of them will fire up Typhon.
23:57
I think if you were them, still use SAS, some of them.
23:59
Put five up a studio. Now we're in the clouds, a few
24:04
of them have got a love affair with the Amazon thing that does
24:09
various magical B. But I think we have hundreds of
24:12
these people. So I you will hear me say very
24:17
often and it scares people. The first time they hear it, I
24:20
Basically don't care what it is, they're using to do the
24:25
experimentation. Because the moment I care about
24:28
that on my side of the world, the universe just ends because I
24:32
got to keep track of all these different things and we're
24:34
stifling Innovation what I care about is that whatever they do,
24:38
can't get out the iron box, and once they do have something
24:42
useful, we can move it back onto our side where we comes back
24:45
under regulation. Damn.
24:48
So this iron box. I mean you've hinted at Tardis a
24:51
little bit, I think we should go into that now.
24:53
That's a great segue into what exactly it is.
24:56
I loved this whole conversation started because of a LinkedIn
25:00
post that I randomly saw where you were talking about Tardis
25:04
and basically I have a very miniscule understanding of it.
25:10
And I really wanted to talk to you more about it because it
25:12
seems so fascinating and it seemed very different but also a
25:16
little bit. If I can use that term as to how
25:20
to do something like this, and let's just start with like the
25:24
basics. What is it?
25:26
And what does it do? Okay, well let's start off with
25:30
the analogy. Are you familiar with the doctor
25:33
who show? No.
25:35
Okay there's a site like oh, you need to have watched.
25:39
There's an old British sci-fi so where the guy would go into a
25:46
little British police box. Box and it would shake and
25:49
Rattle and then it would reappear at some different parts
25:53
of the world and in some different time and through this,
25:56
he could travel through time and he could travel through space
25:59
and each episode, he would typically because it was an old
26:03
British show, he would pick up, pick up some pretty girl along
26:06
the way, but the important thing was there be some new collection
26:09
of bad guys. And he received these new bad
26:11
guys, but the idea was that the tub is allowed you to be the
26:15
same. It was exactly the same in the
26:17
lead. East side but he could 50
26:19
through time and space. The idea behind the Tardis was
26:24
we wanted a way for our IP to be able to travel through time and
26:28
space. Now, what do I really mean by
26:31
time and space? Well, the first one I've already
26:33
alluded to I've been playing this game for 20 years but I'm
26:37
the new boy. We have products that was
26:41
started before. I was born and they're still
26:44
being used to the rate of hundreds of millions of dollars.
26:47
A year. So, you know, we are in the very
26:51
long haul. Here we have customers, we still
26:53
support with a Mainframe in the basement.
26:57
So you know, we're around for the Long Haul.
27:01
But I want, I mean my kids work for the company.
27:04
I want this working in 50 years time, so there is very much a
27:09
view. Oh, and the other thing is, You
27:12
say, you got lots of people who work in startups and that sort
27:15
of thing, we buy dozens of startups.
27:20
And I would guess 2/3 of them. Three quarters of them are going
27:25
through a technology, refresh when we pick them up, and it's
27:28
normally because they go into a technology refresh, they finally
27:31
work out isn't going to work. That's because when they do the
27:35
startup, they go out there and they gather all these really
27:38
cool third-party tools, they glomp them all together, they
27:42
burned them into their code and then two or three years down the
27:46
line. Some of those third-party tools
27:48
are no longer was wonderful as they used to.
27:50
B. I mean just go look at the
27:54
Apache attic and look at Apache. I was at Apex, this was the
28:00
world's best bat system three years ago when till they got
28:03
into a fight and no one ever supported it again.
28:07
So the notion behind the Tardis is we wanted a layer of
28:11
insulation so that I could gather together all of the best
28:15
third-party tools on the market and we could deploy them but we
28:20
could switch between them. At a moment's notice.
28:25
And when I say a moment's notice, I will literally mean we
28:29
have quite a few processes which will be using one set of
28:33
third-party, libraries and test another setting Q&A or three or
28:37
four different searching Q&A, and then it can get deployed out
28:41
using different ones in different verticals.
28:46
That that is how psychotically I take this notion of being
28:50
portable so you probably heard of jdbc and you think all that
28:54
gives you job of portability? Well it doesn't really because
28:57
the Jarvis adverse SQL syntax is different between each of these
29:01
things. And you know if you think it's
29:04
portable, take a piece of sequel through jdbc hitting my SQL and
29:09
try to Port it, the SQL server. Or try to put away from avocado.
29:14
You will die. We have And literally millions
29:18
of dollars sometimes because some of our old kit was burnt
29:22
into a SQL database that no one was supporting anymore.
29:26
Well, with the talent is none of, that's true anymore.
29:29
We actually have a complete abstraction layer in there, so
29:33
the notion of a database is completely separated away from
29:37
the implementation, so we have exactly the same code hitting
29:41
sequels mostly calls in-memory databases Is caches you name it
29:48
and that's really the idea behind the target it completely
29:52
abstract out this notion of your data algorithm from the
29:57
implementation of the data algorithm on any particular set
30:00
of Hardware is Skyler. I lead machine learning and
30:03
health rhythms if you want to stay on top of everything
30:06
happening in ml. Ops subscribe to this podcast
30:12
that Naked. Slower like is it a translation
30:20
that's happening? Is it?
30:22
It's just like as it has to go through that or is that all done
30:26
beforehand? Like what does that look like
30:29
the idea cells where the metadata is they are effectively
30:35
compiled at application load into a set of in-memory data
30:41
structures. So you do take a little bit of a
30:45
hit on application load although Some magic that I can talk about
30:49
if you're interested but once the application is loaded there
30:53
is a little bit of translation, overhead if you actually got
30:58
down on your hands and knees and actually make it it.
31:01
But remember there's two factors going on here?
31:05
One is yeah, technically there is a translation layer so it
31:09
must be costing you something, but I've got a team of a dozen
31:13
phds who were coding for low-level access routines and
31:17
Three thousand people who are now using those access routines.
31:21
How many times do you think those three thousand people
31:24
would have actually written low-level code, as well as my
31:27
dozen phds? I got over here and the answer
31:31
is, it hasn't happened yet? But yeah.
31:34
So in fact, what you find is that you get net performance
31:38
lift because what's happening is your low level access code is
31:42
being written by people who can now spend the time to actually
31:47
get it, right? And the data people focus on the
31:50
data. It's sort of really, really,
31:54
really if one's honest with back in the day, yes, I am.
31:58
That old used to have your it team and used to have your
32:01
modelers and between them. They used to do ml Ops back in
32:06
IBM 370. Then someone discovered the in
32:10
the Big Data world that didn't work.
32:12
So we joined them together in this unicorn that was called a
32:14
data scientist, which was lovely.
32:18
Not many people are actually weird enough to have degrees in
32:21
computer science and math. I am one of them but thankfully
32:25
not many people have my mind. But this became popular.
32:31
So over time, lots of people started calling themselves data
32:33
scientists, but they don't really have those low-level
32:36
coding skills. I mean, just look at how many
32:38
you for me using Python and it's like, you're convincing me.
32:41
You've actually got low level coding skills.
32:43
No, using python ear. I'm not thinking.
32:45
So, so we've sort of split it back up now.
32:50
So do the, he got the people who really are genuinely doing.
32:53
Fantastic work understanding, the customer problem and turning
32:56
it into a data problem. And then we have the people who
32:58
are taking the data problem and turning it into something that
33:01
can absolutely fly on modern architecture.
33:05
And so, one way that I have, I think that you had mentioned it
33:10
before is it's like a jvm but for data science is that correct
33:17
way to think of it in a jvm think like get instruction put
33:22
instruction right to disagree to disk is handled by the jvm, we
33:26
have a data science but for machine, so in that virtual
33:30
machine, you could fetch records.
33:33
You can send records, you can read records, you can write
33:36
records, you can index records. I mean, in fact, the thing is
33:40
grown, even since there's LinkedIn post.
33:42
So, one of the things we found is literally just moving data
33:45
from one place to another takes an astonishing amount of time,
33:49
reading it in a particular format off, disk takes an
33:52
astonishing amount of time. So, we move all of those sorts
33:55
of things down into this data machine.
33:59
So, literally, now when I'm doing data work, I do it on.
34:03
I'm going to pick up this screen if they can.
34:05
I do it on my laptop, I can do huge big.
34:08
What? Actually, on my laptop, with all
34:11
of the advantages of local editing and all of that kind of
34:14
stuff, and then I can just push a button and it goes up into the
34:18
cloud and is now processing. Tens.
34:20
Of terabytes of data using completely different underlying
34:24
implementations but it just works and that is the beauty of
34:29
the data science virtual machine.
34:33
I want to go back a little bit to the point about the iron box
34:36
and understand a little bit what that actually.
34:38
Means and you know, sorry if I missed anything here but the way
34:43
that I understand it is the data scientist as you reported as an
34:47
experimentation period. And once they think they have
34:49
something said it goes into A format that is then converted
34:55
into your specific DSL and and allows you to run this work
35:00
quality and implementation checks that you need to, to make
35:03
sure that it can actually use in production.
35:05
What do you mean by Iron box? Like what does it actually look
35:08
like? Well, I mean these days is
35:11
actually up in the cloud and I, frankly don't know all the
35:16
things, our security team do. I'm I know that they get all the
35:21
deer hundreds of times a year but It comes down really to
35:26
setting up the policies for the subscription in such a way that
35:31
data can only get in and out in Define ways and that the users
35:35
don't have access to change many of the attributes of their
35:40
subscription, frankly, the way you but we sort of describe do
35:45
this to them was laptops in the sky and that is sort of what it
35:50
is. It's their laptop they can Get
35:53
at the data, we show them. And it isn't really a laptop
35:55
obviously. And they have some amount of
35:57
scratch disk, but we follow you squabble when they're done with
36:00
it. And you know, that that is
36:02
really what the iron box is. It's a very tightly controlled
36:06
sandbox environment, where security, and audit and legal
36:09
and everyone's is sprinkled enough.
36:11
Holy water around to convince themselves.
36:13
That data is not getting out of there, got it?
36:16
Got it. So just sounds a lot like I
36:21
guess you could call it like a cyclops or just a general.
36:24
Oh my gosh, you know, I don't know why I'm even trying to
36:26
organize this basically, it's a safe place where once data
36:31
scientist has an algorithm its data, your it and security team
36:35
can make sure that that algorithm can be then turned
36:38
into a high-quality system. Yeah.
36:40
A client can use with the right performance. but then for it to
36:44
get out of that box, you have to basically do a translation so
36:49
that there's that metadata that comes through and so you can
36:52
always go back in time and you have that lineage for it and you
36:56
have the ability to trace it and I imagine also it when you're
37:02
doing the metadata, does it like scrub Pi or is pi already, like
37:07
way out of the question when it gets into the box, there's a few
37:12
different answers to That not being awkward but we it depends
37:17
exactly on the situation. In the commonest case, we have
37:22
something internally called Alexa ID which is a unique
37:25
identifier that we can actually use to tag an individual.
37:28
So you may, remember I said the data comes in from the customer
37:32
it will have had Pi on it but what we do is we take off the pi
37:37
and we put our own internal ID on the encrypted with a one-time
37:41
key so that we can then actually treat that as if it has non pii.
37:50
Now the other thing this me, you've mentioned testing if you
37:54
times, one of the things that we do for the quality side is very
38:00
often. The training data is not
38:02
complete. What do I mean by that?
38:05
Well, suppose I've trained a forest against this data and the
38:10
follow is comes popping out. You've got all these decision
38:13
points in your tree. Now, you know how those decision
38:17
points actually impacted the data you have, but Don't have is
38:20
a couple with metric, if you think of something like Java,
38:25
when you're testing your job or you know, how many lines of java
38:28
actually got tested? You don't know when you train a
38:31
model, how many of those decision points inside your
38:34
tree? Were actually impactful?
38:37
But because we have the tree actually encoded in metadata.
38:41
We can now hop to the other side and actually generate fake.
38:46
Customer data that would exhaustively test the tree and
38:50
push it back. And then say, this is what you
38:53
think you gave us, is this what you actually wanted, and we have
38:57
found quite a few times that has actually identified holes in the
39:00
training data. So that's why I said, it depends
39:04
when it comes down to Pai. Now, we also have a few other
39:12
tricks in the area of differential privacy in a
39:17
familiar with differential, privacy a little bit but I'll be
39:20
home with an explanation from your differential privacy and I
39:26
wrote a paper on this called heavy kneecap Theory back in the
39:30
early 2000s and the idea of how we kneecap theory was that I
39:35
could set up a camera at knee height just to track everyone
39:39
going around the building and I could know what everyone did and
39:43
because the number of hairs on your kneecap is Unique.
39:47
I could identify individuals unique play even though I didn't
39:50
actually know who anyone was because they couldn't identify
39:53
their kneecaps, that is my slightly Whimsical version of
39:58
differential privacy. But it is something that impacts
40:01
us. Particularly once you get into
40:03
fall analytics, you're working out at the end of the bell
40:06
curve. So you have to make sure that
40:10
even things you don't think of as being pii, actually can be
40:15
pii There are first names which on their own are actually Pi in
40:20
this country so we can actually be more private than Pi because
40:29
we have effectively the dictionary.
40:31
We know everyone in the country, we know, for every person how
40:34
little it takes to identify them.
40:37
And therefore, we can avoid anything that would actually
40:40
identify them being attached to that data.
40:43
Yeah. I was thinking about the who is
40:46
it? Elon musk's kid's name is one of
40:49
those. I think that is a is a unique
40:53
identifier just because of how it is.
40:56
So wow. And also I love this hairy
40:59
kneecap Theory. I'm gonna probably tweet about
41:02
that later. It's a great one.
41:05
Now let's jump into this. Other post that I saw on
41:10
LinkedIn and then I'm very intrigued about and it is you
41:13
speeding things up and you if there's one thing I've learned,
41:16
And you are, you're making sure that things stay fast, and
41:23
you're making sure the right people are tasked with the right
41:25
jobs. A data scientist isn't going to
41:27
go and try and speed things up with their python.
41:30
You got to get the phds who can work at that, low level 2 speed
41:35
it up. You, I think from what I gather
41:37
from this story, you were one of those people trying to speed
41:40
things up, right? And you were able to do it by
41:45
breaking the rules or change. In the rules, I think was my
41:47
favorite part of the story, but can you give us the whole, the
41:51
whole context here? Well, there were a few of those
41:54
I put out there so I'm not sure, exactly, which one you're
41:56
referring to was robbing to jog, your memory.
42:03
Exactly anonymized linking. Exactly.
42:06
Okay. Well, let me tell you the bit
42:08
behind the speeding things up. Then we can go into a linking if
42:10
you want to normally one of the things you're thinking about
42:17
when you're trying to optimize something is how big a query
42:21
space. Do you have if you have a very
42:25
limited query space? So people at the simplest.
42:28
If people are only ever going to ask you one of five questions
42:31
you can do all the work to pre-compute those five questions
42:33
and then hand out can't answer. Now when you come in to search
42:39
which is the particular thing I was dealing with their, the
42:43
logic has always been You can't use pre-canned answers for a
42:48
search because there's an infinite number of possible
42:52
search Focus. You know, you go into Google,
42:54
you can type in absolutely anything and therefore you can't
42:57
pre-compute search results and that was static Theory and to my
43:04
sugar in. There were even videos that my
43:06
team insist on playing what I was hammering.
43:08
This into newbies 15 years ago but as I was going through this
43:14
one And it occurred to me that there's multiple types of
43:17
search. There's the search where you
43:20
really want to know about a range of things.
43:22
So, you know, you've gone on to Amazon and you're trying to find
43:25
a gift for your wife or something.
43:28
But there's also the search where you're specifically
43:31
looking for a particular thing and linking and matching is a
43:36
search where you're specifically looking for particular thing.
43:40
Now, the little Epiphany I had was if you're looking for a
43:43
particular thing And we have the universe of people then in fact
43:48
there are limited number of queries.
43:52
There can only be one entity that you're looking for, so I
43:57
can reverse the problem and I can say for every one of my 400
44:01
million people, what are the valid ways that I would let this
44:05
person be searched for, and then I can actually pre can all of
44:10
those answers. Now, if you want to generalize
44:14
that and view it, in terms of a model, you need to know what
44:19
part of your model, is your customer sensitive.
44:22
Tim in the, you think, well, I've got this whole model but
44:26
very often either the ends of the model, they don't care about
44:30
all the middle of the model, They Don't Care About.
44:33
Now for certain, it's even better in the everything to the
44:36
left of your first record. They actually don't care about,
44:40
they don't care. How much they don't care about a
44:44
million other people, but only care about the person versus
44:47
searching for, and therefore, I could affect a feast throw away
44:51
all of those vague responses on all of those.
44:54
Well, no, maybe kind of responses and they only had two
44:57
pre. Can the ones that they were
45:00
actually going to get hits on and that is how I solve that
45:05
particular problem and I mean, the speed up there was utterly
45:10
ridiculous. I got I think, It was an order
45:13
of magnitude reduction in latency.
45:15
I got down to about three milliseconds.
45:17
And we were pumping fully fifty thousand transactions.
45:19
As and we've actually been better than that.
45:22
Now, That's a really fun story. I think what I like about that
45:29
is it sounds like what a lot of people in the data science and
45:38
engineering work with work deal with day-to-day.
45:42
And that is, you know, you're given a very sort of loose
45:47
problem where the demand is for a lot of impressive, technical
45:51
work to some degree, right? And they sort of the sort of
45:55
charges that you're given is, hey, go make something
45:57
miraculous happened that nobody has happened, that has never
46:00
happened before, right? And I think that a lot of people
46:04
start with that approach and say, okay, I'm going to try and
46:07
do develop the best technology possible for the situation, but
46:10
I'm reminded of what Jeremy Howard became on to a podcast
46:14
fast. As I said, a couple maybe a year
46:16
ago at this point which is that if you want to problem, really
46:19
solve give it to someone who's lazy who doesn't want to do that
46:22
for her and they will find a way to define the problem in such a
46:25
neat way that solves exactly what it is.
46:27
You need to solve for and does not much more.
46:31
And that's the way I think. Think that you know I have heard
46:33
in a lot of programming books and in general you know to be a
46:38
much more efficient approach as demonstrated by the example and
46:45
I think this has been a great conversation thus far, we've
46:46
talked a lot about what LexisNexis is done.
46:49
We talked about what the machine learning and data platform looks
46:52
like at LexisNexis our crews were busy use cases, where I'd
46:56
love to take it next to talk a little bit about your experience
46:59
as Data professional for the last 20 years.
47:04
What? Lessons we've learned along the
47:05
way probably for longer as a data professional during that
47:13
time and met. The first place, I'm going to
47:16
start with, is this? What is it that kept you working
47:19
at LexisNexis for the last 21 years?
47:22
No, I'm obviously a geek. but I'm a geek that can impact the
47:29
world because I'm working at LexisNexis I mean, I don't know
47:36
how much you looked into the history, but my very earliest
47:40
work was after 9/11, we actually worked with the various American
47:47
agencies, tracking down another 80 terrorists that were in the
47:50
country that no one had. Well, I'm not going to go to
47:55
some company and start finding ways to sell soap.
47:58
Better. When I actually have a chance to
48:01
keep the country safe. There are literally Real killers
48:05
that will put behind bars because of the technology we've
48:09
created even now it's our systems that are going out
48:13
there, tracking down, missing kids, there were people out
48:17
there who are getting credit. That wouldn't normally, because
48:20
we have systems that can actually give them credit for
48:22
the way they behave. Why would and I mean, if I want
48:27
to be a technical geek, we also have by far the biggest and most
48:30
complicated technical integration job on the planet by
48:34
now. um, so, if you will, if I believe in what I'm doing, This
48:43
is the place to do it. And I mean, I also have to say
48:45
for me, personally, as you probably detected, I'm not your
48:48
typical big company, gray suit. I'm a little weird.
48:53
I do my thing. I'm going to do it my way and I
48:55
want to be surrounded by the best when I'm doing it.
48:58
And Lexus is a big gray company, but they have been compared to
49:03
really build this little environment in which I can do my
49:07
thing and I can do it to the best of my ability.
49:10
And then they do their thing all around me.
49:11
And the end result is just fantastic.
49:14
You know, I love the way that you frame that because I can
49:19
actually attached to some of that impact that you're talking
49:22
about what my company, which is a LexisNexis user, which
49:27
curiously enough occurred after Demetrius.
49:29
Decided to do. This podcast does is work with
49:34
seriously, mentally ill individuals and part of what we
49:37
have to do is figure out where those individuals are and we use
49:40
the stuff that Lexus has to help us find people use public
49:44
records effectively identify where they actually are not
49:48
wherever we get to list it. Us, which tends to be stale and
49:53
I think hearing you talk about your motivation, for sticking it
49:57
out at a place. With such tremendous impact is
50:01
really awesome. In an age where we have seen
50:03
more mobility in technical careers than ever before.
50:06
Right? The average big company Tech
50:08
worker stays at about stays for about a year and a half to two
50:12
years nowadays, and there's a lot of reasons for that but the
50:16
the opposing side of it, which you A little bit is that you
50:20
know the opportunity make a lot of impact work at scale and
50:23
solve interesting problems. It certainly does.
50:25
You know I think Here, an important message and no
50:32
Demetrios us. Something here that I can tell
50:34
from his purpose face that he wants to say.
50:36
So I'm going to take it to him. Oh yeah, I mean this has been
50:40
incredible. We got to wrap right now but I'm
50:43
so thankful David for you to come on here and tell me bunch
50:46
of shit that I had no idea about.
50:48
Like I learned a ton and I also am just like every time I would
50:55
hear you talk, I would get 5 more questions so it's I was
50:58
like, I want to do a round two of this because especially at
51:01
the end there when you're talking about some of this stuff
51:03
and I know it's also of interest to you is the ethics of what
51:08
you're doing and how you're going about it.
51:10
I'm I mean, you stuck around, you've been able to go and, and
51:15
do some really incredible things.
51:16
As you said with, like capturing terrorists or making sure that
51:22
serial killers, get put behind bars and I imagine, there are
51:27
deep, ethical Oil questions that you have to face on a day-to-day
51:32
basis. And and so I want to dig into
51:36
that. I want to dig into more of like
51:39
just how you're optimizing more of your setup.
51:42
The Homegrown solution that you've got and how you were able
51:45
to move to the cloud, what that looked like.
51:48
There's so many good things. So if it's alright with you,
51:52
we'll just cap it here and then we'll try and do a part 2
51:54
sometime in the future found that that's it.
51:57
Thank you for your time.