Latest / Agentic Conversations (formally mlops.community) / How Explainable AI is Critical to Building Responsible AI // Krishna Gade MLOps // Meetup #53

Transcript

How Explainable AI is Critical to Building Responsible AI // Krishna Gade MLOps // Meetup #53 · Agentic Conversations (formally mlops.community)

0:16
All right, so before we start this Meetup, I want to do a few
0:20
things that we don't normally do.
0:23
I'm gonna call out Fiddler AI because they are an incredible
0:28
support to the ml Ops community. Really, and I think it is worth
0:32
mentioning that they have sponsored the community.
0:36
And with that leap of faith, it helps the community grow.
0:41
It helps the community be what it is.
0:44
So I want to thank them from the bottom of my heart.
0:47
They are one of the first sponsors and they're taking the
0:51
chance on us. So that being said, let's get
0:55
into this, talk with Krishna. It was an incredible Deep dive
0:59
on. Unexplained ability.
1:01
And why we need it? What it is?
1:02
All of that responsibility when it comes to machine learning.
1:08
There we go. And then there is another thing
1:12
that is pretty cool about today's meet up which is Krishna
1:16
said he is going to give away some Fiddler schwag to some of
1:21
the best questions that are asked throughout the meet up.
1:25
So if you have any questions today, throw In the chat and
1:31
you're going to enter yourself into the raffle to win some
1:34
swag. That's not a bad deal.
1:37
If you if your, I might ask some questions too, because I want
1:41
some Fiddler shrug, so that's a good thing then.
1:45
Yeah, that's it. Just just kidding asking,
1:50
questions is my job. That's what I'll be doing this
1:53
whole time. So today, in case you missed it,
1:56
we are here to talk about how explainable aai is critical to
2:01
building responsible Ai, and let's start first before, we
2:06
jump ahead with any of that. Let's start with you.
2:10
Krishna, I know that you have a long history in this explainable
2:16
AI world. You were one of the people that
2:21
decided to put at Facebook, why am I seeing this?
2:25
And then later that Trend caught on, we can see it in most large.
2:31
Platforms. You have some kind of explain
2:34
ability that tells you why you're seeing something when
2:38
there is machine learning involved and you also have gone
2:44
off and you said well, you know what, I like that idea.
2:47
So much, I want to start my own company around that and that is
2:51
Fiddler is doing Monitoring Solutions and they're also doing
2:56
explain ability and so that's absolutely incredible.
2:59
But Before we jump into any of that in the talk theme and all
3:03
that, we came here to talk about today.
3:05
Can you just give us some background on yourself?
3:07
How did you get into machine learning?
3:09
How did you even get into Tech? And absolutely, thank you
3:13
Demetrios, thanks for the opportunity.
3:14
Thanks everyone for joining the call.
3:16
And first up, it's our pleasure to be part of ml Ops Community
3:19
its rapidly growing community, so feel free to join and very
3:22
super interesting questions and people from coming from various
3:26
different, use cases in operationalizing ml.
3:30
So, yeah, I mean II know. I'm like an engineer by
3:32
background. I came to the states in 2001 to
3:35
do my grad school from India. And after that, I landed my
3:40
first job at Microsoft working, as a software engineer at Bing,
3:44
which is actually where I got exposed to machine learning.
3:48
So being was building this search engine to compete with
3:51
Google. And one of the things that they
3:54
wanted to do was to invest in, you know, complex machine
3:57
learning algorithms, especially neural networks back Back in the
4:00
day, this is like, you know, early 2000s where they had a,
4:03
you know, very good, like set of Microsoft researchers, you know,
4:06
at the time that, you know, we wanted to be working with them
4:10
to kind of product ice, neural networks, every single layer
4:13
neural networks, to layer neural networks.
4:15
Not very complex, neural networks, but still, you know,
4:17
quite complex to the state-of-the-art in our village
4:20
party house with them said the time.
4:22
So I remember at the time when we were, when we deployed our
4:26
first neural net, I remember the my like our developer manager
4:30
Just basically Express this thing there.
4:33
Oh we just lost control over such quality algorithm because
4:36
until that point we had a very simple configuration file with
4:40
like, wait specified for each parameter and anyone could just
4:43
see like how the search ranking algorithm would work, right?
4:46
But the moment we switched it to a simple to layer Network.
4:50
We just became a black box. So the very first explain
4:53
ability to it wasn't, it wasn't even called him.
4:55
Explain ability was a debugging tool that I and F another
4:59
developer. Build it was called query
5:01
program and I think it's one of the first query debugging tools
5:05
that I think being still uses. It, basically helps you to kind
5:09
of find out why you're seeing a particular search result for a
5:12
bring such quality. So my foray into explain what he
5:15
started. Kind of unknowingly way back,
5:17
almost 15 years ago and ever since, you know, I've worked,
5:20
I've been working in Tech of worked in, you know, various
5:23
different parts of like being like, you know, and the nice
5:27
thing about working on a search engine, is you work in Like kind
5:30
of a cross spectrum of like distribute systems algorithms
5:33
and, you know, scalability you get exposed to a lot of young
5:35
Concepts there after that. You know, I came to the Bay Area
5:39
worked in a bunch of, you know, social media companies, like
5:41
Twitter, Pinterest and Facebook, where obviously exposed a lot
5:44
of, you know, large-scale data analytics of it and and sort of,
5:48
you know, how do you make sense of the data?
5:50
And it is at Facebook where I kind of, you know, life came
5:52
full circle where you know, Facebook were joined Facebook
5:55
around 2016 and, you know, we and the Time by the time
5:59
Facebook We had invested quite a bit in machine learning.
6:01
We had, we had the FB learner system that people were using
6:05
to, you know, create machine learning models.
6:07
It was like, kind of our machine learning platform and news feed,
6:11
which is a team that I was in, was, was a big consumer of it.
6:14
And so, we would build lots and lots machine learning models
6:16
and, and very complex ones in a sparse neural networks boosted
6:20
trees, you know. And at the time, you know, we
6:23
didn't we kind of sort of lost this ability to diagnose these
6:28
questions that would come from, you know, Issues or internal
6:31
employees and especially after the whole, you know, 2016
6:34
elections and all of the other things that came after, you
6:37
know, Facebook, we felt that we needed you needed to increase
6:41
more transparency among our algorithms first internally as
6:44
press, especially, and also, for external users.
6:46
So that's kind of where I said we started working on so I was
6:49
leading a team called feed ranking platform, which is
6:52
providing the infrastructure for the ranking engineers and news
6:55
feed and and so part of our responsibility was to create
6:59
toolkits to To debug and diagnose machine learning models
7:02
and, and help them expand, and explain them to a wide variety
7:05
of stakeholders starting with developers first.
7:08
And we would have questioned would get questions all the way
7:10
down from Zuckerberg. Sometimes, you know, and why am
7:12
I seeing this story in my feed? Or, you know, why is this story
7:15
going viral? And when we, and previously
7:17
before these tools, it would take a long time for developers
7:20
to do this debugging exercise, right?
7:22
And it could be like, you know, there could be a number of
7:24
problems that could be going on. It could be, you know, a bad
7:27
model got ruled out, you know, kind of Facebook at this
7:29
continuous. Machine learning update process
7:31
and or it could be that, you know, someone was in an
7:34
experiment that they were not supposed to be right.
7:37
Like, and so there is a be testing of models happening or
7:40
it could be some bad data pipeline, that's and Pat
7:42
features. So and so a lot of these things,
7:44
you know, helped us, you know, improve the infrastructure
7:46
improved like the sort of the operations of it.
7:49
And so that's that's kind of what I felt at the time.
7:52
You know, to kind of just to continue.
7:54
Your question is sort of, you know, that's when I felt that in
7:58
a lot of work in machine learning to that.
8:00
That point was, how was most was happening more around, you know,
8:03
helping data scientist, create better models, like, you know,
8:07
accurate models, Automated machine learning platforms, were
8:09
coming out in Osage maker, just launched around that time.
8:11
And, and what could be further? No one was actually looking into
8:14
this aspect of, you know, monitoring and debug ability of
8:16
machine learning models. And that's kind of what prompted
8:19
us to start the company. That's great, man, and there is
8:23
something that you said there at the beginning of your journey,
8:27
that I think is fascinating. Only because you were doing it
8:31
in the early 2000s, you said yeah, it was like two thousand,
8:35
five ish, which we launched two layer Network and we had to
8:39
build Curry problem. I do beg.
8:41
You, I mean, you said that this is what's incredible to me is
8:44
that everything was fine when you were doing it behind the
8:49
scenes, but then when you put it out into the wild, it was like,
8:53
whoa, we just lost our baby, correct, it's no longer under
8:56
our control. Correct?
8:58
Hey and especially for the leadership but right, where you
9:01
can just see hey like you know how what's which feature is
9:03
contributing, how much so they can, they can you have do you
9:05
want to have that Assurance in how?
9:07
How is the system working even if it's working fine or if it's
9:10
not working finds? Yeah, yeah, that's incredible to
9:14
think about. So let's move on to explain
9:18
ability and responsibility. I know there are they are Hot
9:24
Topics right now, especially, I think, even more Because the
9:29
social dilemma came out and more people are starting to focus on
9:34
the governance of machine learning and also the explain
9:37
ability and the responsibility of all of this.
9:41
So I think we should probably start just by talking about what
9:46
is explain ability, besides you saying, okay, if I'm a consumer
9:52
of a Newsfeed, I can just click a button and know why I saw some
9:58
are. Tickle right.
9:59
What else is explain ability. Can you break that down for us?
10:02
Absolutely. So I think fundamentally, if you
10:04
think about if you take a step back, the new software artifact
10:09
that is getting operationalized today is the machine learning
10:13
model, you know, for example, you know, back in the day, you
10:17
know, most of the software used to be, you know, like just like
10:20
you know, code that you write and you can actually see what
10:23
you've written, you can debug it line by line and you can
10:27
actually Nate and by spending time at it, right?
10:31
But the new start for software artifact that is going into
10:34
production, is this probabilistic entity or a
10:36
stochastic entity called The Machine learning model and there
10:39
are few aspects of it there that actually make it interesting.
10:42
One is it's a black box. Even a simple, logistic
10:45
regression model. It's not something that you can
10:49
open up the code and read all the rules, right?
10:50
Like, even if even if there's a decision tree you cannot see.
10:54
You know, what rules could invoked when a particular
10:56
prediction was made. So first of all, it's a black
10:59
box entity that you cannot actually open up and see,
11:02
secondly, because it's a stuck at a spastic entity, Its
11:05
Behavior changes, based on the data, it's receiving, right?
11:08
So you may have trained it with certain set of data, and when
11:12
you actually, you know, actually operationalize it, you may, it
11:15
may be getting in our different types of Data sets.
11:18
So therefore, its predictions can be, you know, something that
11:21
you may not like, you know, may not be able to guess, you know,
11:24
even if they can be across a bunch of and they can be spread
11:27
across a bunch of range. So so therefore these two things
11:31
of not like I wouldn't call it non-deterministic but kind of
11:35
you know, it's it's not theoretically not native Mystic
11:37
because it has a finite range but it still kind of non
11:40
deterministic from a human point of view, the way it behaves in
11:43
production and the fact that it is a black box, those two things
11:47
Make it a very hard problem in a hard, hard thing to
11:50
operationalize. Now, if you operationalize it
11:53
without properly testing, these things probably understanding
11:57
how these models. Then there is this whole quote
12:00
unquote, irresponsible practice of machine.
12:03
Learning could come into picture, especially if you are
12:06
models are being used for some high stakes use cases, right?
12:09
So, for example, if someone is using machine learning models
12:13
for issuing loans, right now, we all know that You know, like
12:19
things like jobs, you know, getting a loan approved, getting
12:23
a house, mortgage approved. The critical aspects of, you
12:26
know, human lives and Society right now.
12:28
It can be life-changing and if you get that particular job or
12:30
it can, if you can get like, you know, that loan sanctioned or
12:33
whatever clinical diagnosis and all.
12:35
So for those high stakes, uses machine learning models are
12:38
being applied because, you know, you have a lot of data collected
12:42
and you have a lot more information about the users and
12:45
you want to, you know, provide them More accurate.
12:48
I mean, in the good spirit only, so since you want to provide
12:50
more accurate X, memo experience to the customer and you can
12:53
actually simplify your operations and what not?
12:56
But if you do not actually know what you're doing with these
13:00
models, if you don't understand how the model is working, and if
13:04
you have not properly, you know, validated it tested in monitor
13:07
it, then you cannot be, you know, saying that you have a,
13:10
you know, you're sort of, you know, make your systems are
13:12
making responsible decisions, right?
13:14
So, so the whole concept of responsible AI Is to ensure that
13:18
a particular set of processes are sort of implemented.
13:22
It's a process framework so that you can think about, you know,
13:25
when you're, you know, tip training and deploying machine
13:27
learning models, what set of things you need to think about,
13:30
you know. So, starting from the training
13:32
data set assessing, you know, by season training, data sets, and
13:36
then validating machine, learning models.
13:38
And then, once you validate them, monitoring them.
13:41
And being able to debug when people ask questions and having
13:44
those capabilities will set you up.
13:47
For a responsible way I practice.
13:49
And so that's kind of, you know, how you define within that.
13:52
Explain ability becomes a core concept because if you do not
13:55
understand how the model is working, that's the fundamental
13:59
part of it, right? You know, someone asking how did
14:01
you make this prediction? If you do not understand it,
14:03
then you can be sort of, you know, you really don't know how
14:06
your system is working, right? And and so so that, so
14:09
explainable to becomes the center of it in its the coil
14:12
becomes a Cornerstone technology to actually have in place for
14:16
you to actually be Responsibly I introductions.
14:19
So when you're looking at the responsible AI, Of course, the
14:25
use case of these high-stakes machine learning models is very
14:30
obvious, right? And I know like, I think the
14:33
epitome of this is the there's a something in Florida where the
14:38
judge decides, how, who gets out on bail through to a machine
14:42
learning model, right? That's like, the epitome of a
14:45
very high stakes, machine learning model, and you want to
14:48
be able to explain why it's making those decisions.
14:51
Turns out it was pretty biased. And I think everyone's heard
14:55
that story before, so we don't need to go too far into it.
14:57
But what I want to know is for those use cases that aren't as
15:03
high stakes. Do you still feel like it is
15:06
important? I hand and how high up the food
15:10
chain. Like is it just something?
15:12
Because I think about the people in the community that are trying
15:15
to just get a model out and then you tack on, oh yeah, you want
15:20
to make sure it's explainable. You want to make sure it's
15:22
responsible. I have my own personal thoughts
15:24
on this. Like you shouldn't be putting
15:25
something out unless I can explain and it is responsible
15:30
but I'd love to hear what you think about that too.
15:33
Yeah, so increasingly what's that?
15:34
What's happening is? Use cases that you have never
15:37
thought would have these kind of implications are emerging.
15:41
So obviously we talked about you know, the standard set of use
15:43
cases the credit underwriting, you know, in our clinical
15:46
diagnosis you know recruiting which directly affect affect
15:49
human beings need to be, you know, much companies, you know,
15:52
Finn I need to be putting in Once their practice and we have
15:55
seen all the incidents whether the Apple credit card, you know,
15:59
gender bias, incident or the Amazon recruiting tool, you
16:02
know, kind of, you know, bias incident or in many many
16:05
incidents in the past. You know, that we have you know,
16:08
companies had to roll back to soar had to kind of, you know,
16:11
do PR correction or kind of in a regulatory probes happened.
16:15
You know what, not right. But if you kind of think about
16:18
it, like there is this UK logistics company called eluru
16:23
and they got sued by an Italian code that they were actually
16:26
issuing these, you know, sort of a delivery orders for different
16:30
people at different times and they were actually being biased
16:34
towards against a certain set of people.
16:36
Now, you wouldn't think Logistics as a, you know, in the
16:38
previous as a company that would actually need to think about
16:41
responsibility, I before it. But if you step back, you know,
16:44
we are actually using, you know, all these, you know services
16:47
like doordash uber all the time but there is an element of, you
16:51
know, a are these algorithms working in a fair manner Right.
16:55
Like are these, I have are these basically trying to sort of you
16:58
know you know provide sort of employment in a fair manner,
17:04
probably give it give, you know, orders in a fair manner.
17:07
So there is a hole that mean it's it.
17:08
Once you open up it, it becomes like a can of worms, right?
17:10
You know, you can you can say the same thing about say housing
17:14
market place company, right? So so I think you can see that.
17:17
There's so many hidden biases that you can actually, you know,
17:20
uncover even in, you know, in use cases where you a not they
17:25
do not directly touch human beings.
17:27
Now, having said that if you are just using machine learning for
17:30
like, you know, something that is completely internal.
17:33
That is like literally it's not touching human beings at all.
17:36
It does not have any implication of it.
17:38
Would you still need monitoring explain, buddy.
17:40
I would argue that, you know, you would, if you, if your
17:44
models are critical for your business, right?
17:47
Let's say if your model is actually critical for your
17:49
business, not something you're doing some for your own
17:51
research, you don't you want to know how it's working.
17:54
Don't you want to understand how how it's actually making its
17:57
decisions. I think it makes, it makes it,
18:00
you know, and our experience, what we believe is you can
18:02
actually use these insights to make the modeling model itself
18:05
better over time, right? The ultimate goal of it is one
18:08
is obviously to explain it to others who are asking your
18:11
question, so that you have these in this insights but then also
18:14
use these insights to make the model better, right?
18:17
So now we are seeing this emergence of whole model
18:19
robustness and adversarial, testing the whole sort of
18:22
research Fields emerging where now how Can you use explanations
18:26
to make like the model better? Right?
18:29
And you know, one of the customers would be working with
18:32
is, you know, is basically trying to use like false
18:35
positives coming out of their model and then using
18:39
explanations for the false positives to kind of, you know,
18:42
derive, you know, like sort of synthetic training example so
18:47
that they can train like a dress, that false positives, a
18:49
segment of false positives and make the model better.
18:51
So there are multiple use case of explain body not just
18:53
explaining to Third party but also to further our own sake to
18:57
make the model better. Right?
18:59
Can we dig into that? Use case real fast.
19:01
So the false positive is coming out and then they're going back
19:06
in their training with that false positive to not have that
19:09
false positive again. Correct.
19:11
So I can I can name this customer has a very high-profile
19:14
customer but let's assume let's assume this scenario.
19:17
Let's assume the simple fraud case scenario.
19:19
So you know you have a fraud model it is detecting fraud.
19:24
And it has a set of false positives.
19:26
Now, first of all, you want to know when you actually running
19:29
model in production are about to launch it, you want to know what
19:32
are all the failure scenarios? Which segments of data.
19:35
It's actually not going to predict as well, right?
19:38
And so explain ability and to like Fiddler can help you
19:42
uncover in a, which slices of the data is the model, not going
19:46
to is not performing well on your own train and test sets.
19:50
Now, you can actually also do it in production when you're
19:53
actually, you know, you can say let's I want to run this model
19:55
and observed it over a week or last month and I've seen you.
19:59
These are the cases, my humans have labeled.
20:01
These are all false positives right now from those false
20:04
positives, you as a developer in the past, what you would use you
20:07
would actually look at and detect patterns to see.
20:10
Okay? This is where the model is
20:12
failing. You know, for example, people
20:14
coming from in a transactions coming out of, you know, San
20:17
Francisco from this neighborhood are being classified as fraud or
20:21
transactions coming from. Nevada from this, you know,
20:23
region are being cast. Is divided as fraud or whatever,
20:25
and you try to detect those yourself as a human and then try
20:28
to go and say do we have enough representation of those in
20:31
positive examples and negative examples?
20:33
You know, because they are all misclassifications and can we
20:35
can we can we can we kind of train the model so that they can
20:38
actually learn from it right now that process you know it's
20:43
harder to do when you have a lot of features when your model is
20:46
very complex. When you do not know the reasons
20:49
how the model is predicting, but imagine if you had model that is
20:54
actually Not explainable D, can give you attributions.
20:56
We don't get. These are the reasons why these
20:59
transactions are classified as you know, fraud, but they're not
21:03
fraud, right? And so now, you can actually
21:05
group those attributions and from there, you can have these
21:08
clusters of attributions which will give you these, you know,
21:11
Common sets of reasons which can then, you know, help you, you
21:14
know, come back and say, okay, what, where do I get, you know,
21:17
what's the problem? Is it a feature?
21:20
Do we have good features for these set of, you know, people
21:23
do. We have enough training?
21:24
Samples right? Or is the model, not good
21:27
enough, you know, for these type of things.
21:29
So we may have to train a different model architecture.
21:31
Maybe we have to go for our instead of one single model we
21:34
have to build like an ensemble of models.
21:36
So those are the things that you can actually do it and this is
21:38
very common practice in, you know, in like Union companies
21:41
like Facebook, you know, where we would basically when we try
21:44
to look at a model performance, we would look at it holistically
21:47
but also look at into into segments of it and when we in
21:50
friend and you can use the same model may not perform well
21:53
across all regions. It's very common for people to
21:55
build these ensembles of models so that it can address like the
21:59
weak spots within the data and then they all come together to
22:01
make a decision. Right?
22:03
So explain ability helps you kind of, you know, get these
22:05
insights, you know, do this at scale instead of a human going
22:08
sifting through the data and, you know, collecting patterns.
22:11
Yeah, and it feels like if it is a human doing it, there's a lot
22:16
more room for error. Okay.
22:18
And a lot more, it's almost like bias could be introduced or
22:23
theories could be. Ooh, stand.
22:25
Then they have no basis in reality.
22:27
Yeah, I mean, you have to have an expert human being.
22:29
I mean, you machine learning is part art, part science.
22:31
That's the reason right. You know, you have this, you
22:32
know, a machine learning. If the subject matter experts
22:35
who some of them are in have this really good intuition of
22:38
what would work, but not doesn't work but what will happen is you
22:41
know, tools like you know, explain ability to is can
22:43
actually you know provide these insights for a wide variety of
22:45
people show that you know, a lot of data scientist.
22:47
Lot of developers can actually, you know, take advantage of it
22:50
and then you know, make sense of make sense of how do they how do
22:52
they address the mental model will and whatnot?
22:54
So that's, you know, I think, I mean, I believe that most
23:00
humans, you know, want to do the right thing.
23:02
Especially the developers who want to like, you know, are
23:04
doing machine learning. And so there's no like people
23:05
are not purposefully gaming system, right?
23:08
But there could be, you know, issues where they make errors,
23:12
you know, you may have not, you know, you may have not looked at
23:15
the training data set. You were not looked at if it's
23:17
actually having enough like samples for each type of user or
23:21
each type of case and whether that's actually bias, I think
23:25
that could actually affect your kind of use case from a bias
23:28
angle or in another way, it could actually just affect your
23:31
model from a performance angle. Right?
23:33
In the fraud example, in which not being will not be biased,
23:35
it's just it. It's Miss predicting, you know,
23:38
transactions, coming from Two Cities, you know, to be like
23:41
fraudulent and and that it's just probably a performance
23:44
thing performance Improvement problem.
23:45
Whereas if it's happening with like it's affecting, like a
23:48
whole gender, like a whole set of population coming from a
23:51
certain place, you know, to be like all fraud, then that Could
23:54
that would just translate into the bias from it's the same
23:57
thing but you know you know, depending on the context it
24:00
becomes like a performance issue or a bias issue.
24:03
So do you need this subject? Matter expert or a someone who
24:10
is very Advanced to be able to set up what the explain ability
24:15
tool should be looking for at first?
24:17
Or is it something that's plug and play?
24:19
And it will know when it like, latches onto the model.
24:24
So the goal is, I mean, the goal for companies like us to fulfill
24:27
it relates to provide as many insights as possible.
24:29
So that, you know, we, you know, we can actually make like data.
24:33
His life easier, right? So you can actually, by
24:35
developing provide your debug ability to these things, even if
24:38
you know, not just the export data scientist.
24:40
But also like, you know, more Junior data scientists on their
24:44
team can also learn from these insights, and make start making
24:47
models much better, right? So that's that is the goal.
24:50
And eventually, you know, right now like our platform supports,
24:53
you know, for example, you being able to like slice and dice data
24:56
and explain, you know, parts of the data with your model but
24:59
also, you know, in the future we may be also like Looking at, you
25:03
know, like kind of said, you know, recommending sizes, you
25:06
know, these slices are actually not performing well for your
25:08
model for now, you and sort of automatically providing
25:11
recommendations. So you can go and actually look
25:13
at those pop, those flash those slices and actually address
25:16
Performance issues and that is super cool.
25:21
I see some questions coming through in the chat.
25:23
So there are some people who want some schwag.
25:26
Remember if you asked a great question, you are entering to
25:30
win some Fiddler AI schwag Big Krishna has been kind enough to
25:35
give out some schwag to select few.
25:38
That ask the questions. Now, let's see here.
25:41
This is an awesome question from shabby, can we measure, explain
25:46
ability where measure the accuracy of the explanation?
25:50
Do you think that we need such metric to regulate sensitive
25:54
models? Yeah, so very good question, you
25:56
know, evaluating explain ability, algorithms is a very
26:00
hot topic and and research today.
26:03
You know, first of all, what what are the things that you
26:06
want to evaluate on, right? You know, there is you know,
26:09
there are many things, right? So for example, is the one of
26:13
the things that you want to care about is is the explanation High
26:15
Fidelity? You know, for example are you
26:18
making up an explanation or is it actually a high fidelity
26:22
toward the model is actually trying to do, right?
26:26
And so that's very important and then is the explanation
26:29
intelligible, you know, is it actually for me as a favor for
26:32
Human on I understand it, right? And so, so there are various
26:36
ways to evaluate it, you know, we are also sort of in working
26:40
on, you know, coming up with own metrics.
26:41
One of the things that in that is actually shining through the
26:44
explain will go. Algorithms is some of these
26:46
algorithms, especially the ones that are coming out of the class
26:50
of attribution based algorithms. So they all come to this family
26:53
of shapley value based techniques.
26:55
And so the nice thing about the shapley value based techniques
26:57
is that they have this sound, you know, game-theoretic
27:01
axiomatic principle. So they Have, you know, certain
27:04
axioms that, you know, theoretically proved by, you
27:06
know, this sort of economists in 1950s that actually help us, you
27:12
know, for example, you know, Ash.
27:14
So I just maybe getting into a little bit of inertia values
27:17
here. So shopping values, was
27:19
essentially designed to sort of thing in Game Theory, to
27:23
basically take like a, a no profit of a game and split it
27:28
into rewards among the players. So it's so, for example, if you
27:31
only say three people come together, start a company.
27:33
In the company starts making profits, how do you take the
27:35
profits and split them into bonuses?
27:37
Now, it's being applied, the same concept is being applied
27:40
for machine learning models, where, you know, a bunch of
27:42
features come together and they make a prediction.
27:45
And now, you take the prediction prediction is your Prophet.
27:48
Now, you take it, and split it into Hmong, in the future
27:51
Rewards or attributions. So, these are all called
27:54
attribution based algorithms. Now, shapley value, shapley came
27:57
up with this really nice. Axiomatic principal saying that,
28:00
you know, two features are actually, you know, I Identical.
28:03
Then they would, then they would get symmetric shape leave or
28:06
they will. If they are symmetric, then they
28:07
will get identical shapley values or if a feature is a, you
28:10
know, as a no op, then it will get a zero shapley's value, or
28:13
all the shapley values, add up to the total, you know,
28:16
prediction whatnot. So the sum of these axioms, you
28:18
know, help us, you know, kind of, you know, provide a solid
28:21
grounding. So we see that shapley value
28:23
based techniques are actually becoming kind of almost the gold
28:26
standard in terms of explain ability.
28:28
There are a bunch of techniques that build on top of it, you
28:31
know, this integrated gradients. It is which is like a variant of
28:35
rapidly values can almond, shapley value.
28:36
Technically, we have our own implementation of shapley values
28:39
using non Theory. So bunch of techniques that
28:41
build on top of this core concept that provide the sound
28:44
principles. However yeah, we still need to
28:46
ensure that. You know, how how is this metric
28:49
right? Like what how you know what is?
28:51
Is this explanation? Can I trust this explanation?
28:54
So today what tools like Fiddler provide is ability to do what if
28:57
analysis so that the human can themselves SS?
29:00
You know you know like what if I change It.
29:03
Now what if I change this feature value to this value?
29:06
What would be my explanation or what would be my prediction?
29:08
So therefore by interactively playing with the model you can
29:12
actually understand it much better.
29:14
And so so, so so they're basically sort of those are the
29:18
experience of the user experience is kind of trying to
29:20
come together to kind of make it like more trustworthy.
29:23
But yeah, in the future we would have to have these kind of
29:25
metrics that that can, you know, can can quantify these the value
29:29
of explain building. Well, that just, we can jump On
29:33
the next question that's coming through which is another great
29:36
one from Sunday. Yeah.
29:38
Forgive me if I pronounce the name wrong but it's it goes
29:43
which we can just pin this on to that because they're asking does
29:48
explainable AI. Also, entail explainable data?
29:52
I see the point where explain ability can help with getting
29:55
the insights about data after the model has been trained but
29:59
should it be handled perhaps more proactively where you Bias
30:03
the data before training the model on it.
30:06
That's a very good question. In fact, one of the things that
30:10
we know, we actually held upon the tool provides to our
30:14
customers today, is it. Also, when you are doing feature
30:17
selection, when you have a lot of features within your data,
30:21
you can use explain ability, not just after the model is created.
30:24
But before it to actually see, which features are actually in,
30:28
on the most important features, which features can be pruned
30:30
away and to Sunjai has Point around Like preliminary
30:34
investigation of feature importance and you know which
30:37
features are coming up. Are there any like looking at
30:39
feature correlations and, you know, PDP like partial
30:41
dependency Place between features and predictions.
30:44
So, you can get some idea before even go deeper into feature
30:47
engineering and, and model building exercise.
30:50
So, there is definitely a valid point there.
30:53
However, one should remember that explain ability is always
30:56
only giving you a lens to look into the model.
30:59
So even if you're explaining the data, you're looking through the
31:02
models eyes. He's right.
31:03
It's not causal from a data point of view, right?
31:06
So it's not like for example it's not going to tell you like
31:10
if for example if a store is not going to make sales because of
31:14
an external event, like coronavirus has happened and
31:16
people stopped coming to the store, right?
31:18
You cannot like you that those things cannot be captured.
31:21
So so explain ability is limited and it's very important to
31:25
understand. This thing.
31:26
It's essentially proving you giving you the eyes to look
31:29
through the model. And the model is then model is
31:32
basically what is the model? You basically learn from the
31:34
data and developed a pattern from the data, right?
31:36
So you're essentially looking looking through the models eyes
31:38
and on the data. So that's what you're doing with
31:41
explain. Really.
31:41
So so there are, you know, you can stretch this thing and get
31:45
into like causation and all which is has to be, you know,
31:47
sort of, you know, little bit careful, you have to be before
31:49
you do that. So it's still, you know, you're
31:51
still going through the model size to understand the data.
31:55
Man, I love this shit. This is so good because this
31:59
needs to be happening, right? Like these questions need to be
32:03
coming up and I argue for everyone being able to explain,
32:08
what is their model doing and why?
32:11
And all of that. And I'm glad to see that.
32:14
Others are asking questions. Maybe it's just because we're
32:18
giving out some swag, but I think there's some genuine
32:21
interest here because there's some incredible questions.
32:24
We've got one from Depeche, who's asking as a data
32:28
scientist? There are situations when the
32:31
prediction output is expected to support a business decision
32:35
taken by senior Executives. In that case, when the
32:40
explainable model gives out a prediction that doesn't align
32:45
with the stakeholders expectations.
32:46
How should one navigate through this tricky situation?
32:50
That's man, explained really becomes your friend Deepak.
32:53
So because That's actually that the hippo decision-making, the
32:57
highest paid person, you know in office, right?
33:00
Hip has called hippo decision-making.
33:02
So-and-so that exists in many Industries it and so and it's
33:07
you have so. So when when you are models
33:10
decision mod prediction, does not match with this.
33:12
You know, senior leadership decision.
33:14
You can then go and explain here is why that is the case because
33:17
they generally would not naturally ask, you know, hey,
33:20
like I don't think my intuition tells me this or my experience
33:22
tells me this, right? But you can actually open up up
33:25
like the model and roquet. Like tell them in know and they
33:27
can actually play with themselves.
33:29
For example, let's take this simple scenarios.
33:31
It's like I'm in a very experienced underwriting officer
33:34
in a bank. I'm like 20 years of experience
33:36
underwriting loans. I know what makes a good loan.
33:38
What makes a bad loan. Now, suddenly a new loan comes
33:41
in and it's marked as, you know, very high risk.
33:45
And I think like a know, it doesn't look like I risk.
33:47
I have approved these type of loans in the past and why is the
33:49
model saying, is high risk. Now, if you can actually explain
33:53
that, you know, saying that, hey, the reason Why is because
33:56
this person's you know loan amount is shown so or this
33:58
person's you know FICO score is zones.
34:00
So you know his previous data stones on what not but and then
34:04
also provide more examples, you know?
34:06
Hey these were all the other loans that look like these loans
34:08
that we actually denied in the past and they were actually bad
34:11
loans or and or we actually prove them but they turned out
34:14
to be bad loans or where they can you can provide some base
34:18
for the person to actually you know change like variables and
34:22
make it make sense of their own self.
34:24
And so this office Really becomes like a collaborative
34:26
exercise. Now, in some cases, the model
34:28
could be wrong, right? And this is where explain
34:30
ability becomes important because, you know, when you are
34:32
designing the model, if you have not selected the right data set
34:34
and whatnot in, in the subject matter expert can actually be
34:37
right and they might be able to guide the model building process
34:40
in cases where the model is right?
34:42
Then they can actually correct their own into in kind of
34:44
intuitions so it brings both parties together and that's the
34:47
beauty of it. And by opening up this black box
34:49
you actually not just like have like the a scientist telling
34:52
this is what the model is doing. And in the past that used to be
34:54
the case, right? And the past.
34:56
But when I looked at the design is, when I would asking what's
34:58
this prediction and they would be like, I don't know, the model
35:01
said, so right. So so that that both like kind
35:06
of know takes away the credit, right?
35:08
And then we understand as an engineer or understand.
35:10
Yeah, it's actually hard to understand, ask the answer that
35:12
question, but, but, but having that opportunity to expose that
35:16
inside, you know, brings multiple parties together.
35:18
We can have to have a dialogue and have a consensus to move
35:20
forward. Great answer.
35:23
And this is a bit of a tangent but I know that when we as
35:29
humans they did a study on us when we see something that comes
35:34
from a computer or a machine, we are much more likely to believe
35:41
that it is true. And so you get into these
35:44
situations like oh well the model said it was this and so
35:47
you just take it for granted without actually digging behind
35:50
and saying, is it right? I mean, I'm sure the people on
35:53
this call, they don't just take everything with without a grain
35:58
of salt. But for the majority of people
36:01
were used to calculators putting in numbers and spitting out an
36:04
answer and it is the right answer always you know and if
36:08
it's the wrong answer is because of something that we did.
36:10
It's not the other way around where the machine learning model
36:12
is just doing something. And then we get an answer and
36:17
we're not sure if it's right. So that's a side note.
36:21
I've got It's more questions rolling in here, there's one
36:24
from Sherry asking, how do you differentiate explain ability
36:27
from interpretability? And how important is each in
36:32
what context? Yes, I mean these are all
36:35
obviously industry terminology right now.
36:37
The kind of the definition that is becoming more popular.
36:41
Is I can at least talk about, think about it.
36:46
Interpretability as interpretable machine learning
36:48
algorithm. If the model itself is
36:50
interpretable, for example, Well, let's say, you know, you
36:53
can one can argue that a decision trees in reputable.
36:56
If I can visualize the decision tree, I can see, you know, which
36:59
path, you know, you know, a prediction is taking, you know,
37:02
I can actually understand. I don't need a separate
37:04
explainer for it, right. You know, same thing, you know a
37:06
linear model could be interpretable because I have the
37:08
lean of the weights and coefficient so I can actually
37:10
have integral. So you have this access of
37:12
interpretability where you know the model itself is
37:14
interpretable MM you know maybe in the case of a linear models
37:17
they or a decision tree to like completely black box where I
37:21
cannot lie No, interpreter neural network explain ability
37:24
is more around, kind of, you know, taking a model and then
37:28
explaining it in are mostly kind of, you know, in the context of
37:31
a post-hoc explanation, right? So in the case, in this case,
37:33
the model itself is not, you know, interpretable and, and
37:37
you're trying to explain it to sort of, sort of a third party,
37:42
or for yourself. And so that's kind of how I
37:44
differentiate it. So you have interpretable
37:46
algorithms, you know, you have the, you know, all the way from
37:49
linear models decision, trees Gams.
37:51
You know, which are, you know, if we were getting being very
37:54
popularized, the all the way to like, you know, like non
37:56
interpretable algorithms like neural networks.
37:58
But then you have explained ability, algorithms, which are
38:00
taking any of these models, and trying to explain it in a way in
38:03
which we talked about shapley values that I am, and, you know,
38:06
integrated gradients and many other algorithms that are coming
38:09
up, that's kind of how I differentiate it, okay?
38:13
So there's another one from Fatima, and they're asking how
38:18
useful do you think caused or ml can be To the explainable AI
38:23
agenda. Yeah, so I think, I think, I
38:28
mean, a lot of, you know, I'm not an expert in causal machine
38:31
learning. So, from what I understand,
38:33
obviously if like, you know, followed Judea Pearl and, you
38:35
know, sort of, you know, read the, the hold the Holy Grail,
38:38
The Book of Y and also, I think, you know, causal ml is all
38:42
about, you know, you sort of you're trying to serve as a
38:44
human providing a lot of structured information to the
38:48
model in all kind of, you know, the sort of the the
38:50
relationships between the elements of the data, a priori,
38:54
but from your own subject matter, expertise and Sort of,
38:56
you know, use in baking that into the model, right.
38:59
So I think that's, that's great. If you can do that in, oh now
39:02
you can actually help into it will help you sort of, you know,
39:04
understand how the model is working.
39:06
But however, it's really hard, you know, in practice to be able
39:09
to do that right for a designer to come together and actually
39:11
associate in order to do the sort of these sort of, you know,
39:15
put to put together these definitions in manual.
39:18
Because what's happening is what the reason the reason why
39:21
explain ability is becoming so popular is because of the
39:24
emergence of deep learning right now.
39:25
What deep learning. Has done.
39:26
Is it's basically taken that you know you have all of your all of
39:31
your data just throw the data into the model.
39:33
Let the model figure out the feature combination, let the
39:36
model kind of you know, train like you know you know which
39:38
pixels are the most important pixels or which word
39:41
combinations make me come together to make like the
39:43
sentiment of a tech piece of text or which features in a
39:46
structured data sets can come together to make it more to make
39:50
it like you notice a fraud or non fraud.
39:52
So, so that in those cases the model is actually deciding those
39:55
patterns and it The using, you know, and it is human is just
39:59
basically supplying the input, right?
40:00
Supplying the data, almost like a raw data set and and
40:04
therefore, the and so, there's no sort of, you know, you're not
40:07
sort of sitting and designing these relationships yourself.
40:11
And so, this is where, you know, explain abilities become more
40:13
important because if these deep learning models being developed
40:16
by users because they're getting very high accuracy and you know
40:19
then now like as at the cost of, you know, interpretability and
40:22
so therefore you you want to kind of address that.
40:25
So that's kind of where We are seeing the usage, increased
40:28
usage of explainable deal. We got more questions coming
40:32
through, in the chat. Looks like people want some
40:35
schwag. We didn't even tell him what it
40:37
was going to be. I what?
40:38
If it's such a letdown at the end, we just say, all right,
40:41
we'll send you a sticker, all right?
40:44
It's not going to be a Justice sticker.
40:46
That is for sure. Krishna said it was going to be
40:49
some cool and I trust them. Now Phillips asking you've been
40:53
talking a lot about decision. Trees are methods like Born
40:56
Again trees, which aimed to approximate models by decision
40:59
trees. Trait traceable for complex
41:03
models or and this I guess I'll just tack on a question to this.
41:10
Are there certain models or methods that are harder to
41:15
explain than others? Yes, I mean, for example, even
41:21
like if you have a hybrid neural network that can take, you know,
41:25
multiple types of inputs, right? Like, for example, let's say
41:29
you're trying to, let's take that, you know?
41:33
Credit card fraud example, right.
41:35
So you have a credit card fraud example now along with your
41:39
structure data sets over example, my you know like
41:42
previous credit card debt and how often I was repaying and
41:46
whatnot. You know, you can also take a
41:49
sequence of, you know, all of my previous transactions all of my
41:52
previous, you know, you know, sort of sort of like instead of
41:58
a as a sequence instead of like a structured data set.
42:01
So you can build a hybrid model where certain set of inputs are
42:03
like more like kind of a tabular data like inputs certain inputs
42:07
look like you know sequences and so now you can build like l
42:10
stm's or you know, but kind of inner query, complex models
42:14
complex and sambas. Now, now you have this layer
42:17
dance Humble how do you then explain it and you want maybe
42:21
explanations that you know different, you know, piecemeal
42:24
levels, right? So this becomes a very hard
42:26
technical Challenge and we are seeing the emergence of these
42:28
type of models come up. And then obviously reinforcement
42:32
learning, which is also kind of, you know, it's actually the
42:35
where the model is actually all the online model learning model
42:38
algorithm. So the model is actually
42:39
continuously retreating. Those are models that are hard
42:41
to explain in practice and and so those are the types of models
42:45
where we feel like it's Spritz. I said this on these lot more
42:49
sort of research has to happen, and then there's like explain
42:52
ability today is like a little hard to apply.
42:55
And and so I think the basic techniques Remains the Same.
42:59
It's about basically being able to apply them at various
43:02
different levels. And how do you then apply to
43:03
their door to get the model that is changing constantly?
43:06
Is there is there is the hard problem and then the other
43:09
question was, Who's there? Another question on decision
43:13
trees correct? Yeah so I don't I don't know.
43:16
The other type of decision tree that was mentioned.
43:18
So I mean the most popular decision tree like obviously
43:21
that's in practice is the booster decision tree for almost
43:24
all and unstructured data sets that people have today BDT has
43:29
become kind of the state of the art, right?
43:31
So it provided it provides really good accuracy, and an
43:35
instant more lot of people use it.
43:38
And so I'll explain ability works very well for those four.
43:41
weboost addition reason, in fact, sharply value based
43:44
algorithms work really well for motion decision, trees in
43:47
practice Alright, so here's another one, a bit in the weeds,
43:54
but from Irv asking how useful are in this is how den and G
44:03
clustering for data, explain ability.
44:06
Yeah, so I mean, I think this probably coming to like, you
44:10
know, giving like given a data, you know, how like, kind of in a
44:13
building like a, some sort of a hierarchical clustering on the
44:15
data to understand the data itself, as I said, you know,
44:18
data explainable Is around in a given.
44:22
My data set, you know what features are important.
44:25
What feature correlations are, you know, coming through, which
44:28
features are actually some, you know, are sufficient.
44:30
So you can actually prune away some more features and save some
44:32
costs. If it's like, you know, very
44:35
high like a large, like slash data set that you have to train,
44:39
or you like bias related questions, you know, which which
44:42
features are coming together to introduce, you know, bias with
44:44
the model. So all of those things are
44:46
various different approaches to do it.
44:49
One approach that we do take with Fiddler is to train like an
44:53
interpretable model on on, on the existing training data and
44:57
then exposing these insights through explain ability.
44:59
So that, you know, it's basically, you know, you know,
45:03
you bring us the data set in Fiddler's, you know, models
45:05
explain, explain the data set by training the model.
45:08
Again, we are looking through the looking at the data through
45:10
the model size. And the reason why I think this
45:12
is actually more effective than maybe a simple, I think a string
45:16
bass techniques which are because these are in Opinion,
45:19
please. This is actually kind of
45:21
modeling the problem that you're trying to solve, right?
45:23
Anything in case, if I'm actually trying to come up with
45:25
like a churn prediction problem, and I want to know what are the
45:28
features that are important to I do and what are the features
45:30
that are not important by building a first pass model?
45:33
That actually tries to predict John with reasonable accuracy.
45:37
You can then understand how the, you know, what features are
45:39
variables are important. What features are not important,
45:41
how different features are coming together to influence the
45:44
production and how the model, how the, how is, like the Are
45:49
distributed across different segments, segments of
45:51
population, how the features of distributed and whatnot.
45:54
So that's kind of what we found more effective in practice than,
45:57
you know, going down the Amor unsupervised way of analyzing
46:00
which are, which is, I think is also useful but I think we found
46:02
this is actually much more much more effective.
46:05
Yeah, that's really cool. So I'm going to take one real
46:08
fast and I'd love to talk about the challenges that you see
46:13
right now with explain ability. Is it the what we touched on
46:18
before? There are some models and there
46:20
are some methods that are really difficult to explain or are
46:24
there other things that come jump out at you?
46:28
Yeah. So I think there are, first of
46:29
all, there's a reason explained in the multiple challenges, one
46:32
is explaining complex. Machine learning models is a
46:36
challenge, the other Big Challenge from a practical
46:39
standpoint. And this is something that we
46:41
sort of sort of working on it Fiddler and and kind of, you
46:44
know, I have solved quite a bit is building a general purposes.
46:48
Explainable quick, swing party platform is really hard, right?
46:52
Because today, unfortunately, models, come in all forms or
46:55
shapes all kinds of formats, you know different formats don't
46:58
agree with each other. You know, tensorflow, pie,
47:00
thought, scikit-learn, extra boost, you know, and have all
47:03
the old-school formats like our SAS and all.
47:06
How do you take all of these models and then build a
47:08
general-purpose explain multi-platform from a technology
47:12
point of view. It's hard because you know,
47:13
ingesting the model and running the model and actually inferring
47:16
the model and different. He is in the model.
47:19
So one of the things that we were excited about was this
47:22
whole open neural net exchange project that came up with him
47:24
and came three years ago which was trying to standardize model
47:27
formats. I think it was started by
47:30
Microsoft Facebook, Amazon as a Consortium.
47:33
And so when we if you have such Platforms in the future it's
47:36
almost like a Java bytecode for models, right?
47:38
So if you actually have like a, you know, when standard format
47:41
for more models than in do to help, you know, for in the
47:44
future for a lot of data science teams to have like a general
47:48
purposes, Elena breathing. So that's that's number two.
47:51
Number three is obviously, we talked about the the sort of,
47:55
you know, the evaluation methods, right?
47:57
So how do you evaluate explain ability?
47:59
How good it, how good is unexplainable team algorithm or
48:02
the other explain volte algorithm that still, you know,
48:04
a topic of, you know, research and sort of people are, you
48:08
know, looking into it. So those would be the kind of
48:10
the three things that I would say.
48:15
So there's a few more that are coming through here.
48:17
In the chat is explain ability only for humans.
48:20
For example, machine learning predictions can bypass physics
48:24
itself like this model, which predicts Planet positions
48:29
directly from data. Without Newton's Laws is
48:33
Humanity. Losing something here, when
48:36
gaining something else and are we only trying to help us feel?
48:41
We understand how the model works?
48:44
Yeah, it's very interesting one. That's a very deep one.
48:48
So I am not thought about it and watch but I think, you know,
48:52
there is always a human in the loop with machine learning,
48:55
right? I mean we have not gotten to a
48:57
state where systems are completely automated end-to-end
49:01
where you know, the systems are Auto learning from their
49:04
mistakes. So even there, I believe that,
49:09
you know, once you have any like once you have this flywheel
49:11
where you have a model that is trained, And you have a model
49:15
that is deployed. And it's now, you know, being
49:17
monitored. Now if you can actually, you
49:20
know, automatically retrain the model kind of, you know, sort
49:23
of, you know, keep it like an almost like a cruise control
49:25
right? Where, you know, it's
49:27
essentially it's just on its own and you and the system is
49:30
retraining better versions of the model and even there to come
49:35
from like the old model to the new model, it requires insights
49:38
right. You know, how do I know where
49:40
let's save the model? You know, you may want to
49:41
retrain the model because the model is drifting in
49:43
performance. It's throwing lot more, false
49:46
positives, or whatever reasons. And from there from that,
49:51
snapshot of like model M 0 to like go to M1, you know, you
49:55
need, you need like explain ability.
49:57
You know, whether the machine is consumed.
49:59
In this case, the machine is probably consuming the insights
50:01
and saying, okay, I these are all the segments of data where
50:03
in the model is not working well and maybe there's like a
50:07
workflow where I magically get like some synthetic training
50:10
data sets, right? So and and sort of automatically
50:13
retrain them. Water.
50:15
But I believe that before we go, there will there always be human
50:18
in the loop because at least, you know, we want to see like,
50:20
you know, which were false positives and then you can go
50:22
and, you know, sort of in a human goes and looks into it.
50:25
And and there is like and they would they could they would then
50:29
go and likely see how they can actually retain a better model
50:32
for putting into production. So, but yeah, I think whether
50:35
the human is in the loop are not in the loop.
50:37
You still need the inside because ultimately explain will
50:40
teach to is to find bugs or performance.
50:44
Problems within the model right? Or explain the model now, so
50:46
find issues within the model so you can actually make it better.
50:49
That's the that's the whole point of it.
50:51
In the end of the day, I want to dive into the idea that you
50:56
presented earlier before. This question and shout out to
51:00
Ron on that question. That was that was one, I did not
51:04
see coming. So the idea of the difficulties
51:10
that you were talking about in the challenges and how to make a
51:14
Tool that is able to span the whole Spectrum right now because
51:18
there are so many different use cases and there are so many
51:21
different models and Frameworks and deployment and all of that.
51:27
How do you aim to conquer that as a company like?
51:31
Yeah, that's the challenge that we set up a set ourselves,
51:34
right? Like know, when we started the
51:36
company, we wanted to build a general-purpose explainable, a
51:38
platform which meant that we should be able to digest, you
51:41
know, multiple types of models and then be able to In them.
51:44
So, so far we have gone quite far in a we can actually digest
51:48
your tensorflow type. I thought scikit-learn extra
51:51
boost type of models, you know, pretty quickly.
51:53
And now we are adding support for ML flows Park, you know, and
51:56
then you know, other type Edge, 20 type of models.
51:59
One of the things that has really helped us is the
52:01
container technology maturity writing.
52:03
You know people have now moving towards more and more kind of
52:06
talk Rising their models. So and so that ability to kind
52:10
of have a Docker being one level of abstraction.
52:12
So do you so instead of You know, we looking at the model
52:16
artifacts we can be looking at, you know, dr.
52:18
Ice containers with a standard API.
52:19
That is implemented. So then, essentially, we can
52:22
standardize or explain what the algorithms to just work on that
52:25
API. So, as long as you can actually,
52:26
give us in. If we can pass inputs, and you
52:29
can give us inputs, you can you can send us outputs.
52:32
Then we can basically explain that model today, right?
52:35
So so those are the things it still, you know, a lot of work
52:39
needs to be done here. As I said 444, like applying
52:41
complex plane will be algorithms like integrated grade.
52:44
You need much more access to the model, you know?
52:47
For example the gradient vectors of the outermost layer and
52:49
whatnot in which case you know you you need like the model
52:53
artifact, right? So in in that in that case, you
52:56
know being able to take the model artifact and run it and
52:59
then run it across, you know, different versions of tensorflow
53:01
and what not is a huge challenge.
53:03
So we kind of, you know, have built the system to scale for
53:06
that. It's a hard problem to solve and
53:08
you know, come very far from where we were two years ago but
53:12
still like a lot of Way to go. Nice pie chart, is that right by
53:18
Dodge to? Yeah, we have a partnership with
53:20
Facebook AI. So there is, you know, Facebook
53:22
has this Library called Captain, it's pretty neat library to
53:27
explain why Turkish models. So, Fiddler uses Captain as a
53:32
internally to kind of, you know, explained by Dutch model.
53:34
So yeah, you can use Fiddler for I thought.
53:36
Yeah, I would hope so considering your background
53:40
correct. Yeah, we have a good partnership
53:42
with Facebook, rehear, research and we, you know, we're two bike
53:44
working towards, you know. Imagine lighting.
53:47
All right Community it has come to the point where we need to
53:50
choose the swag winners Krishna and I are going to look through
53:56
and we're going to tell each other in the chat who we think
54:00
should win the schwag for everyone that is here with us.
54:04
I really want to thank you for these incredible questions.
54:07
This was awesome. This has been super insightful
54:10
and I love hearing how you all are looking at things and what
54:15
is Important to you, really like, and judging by the quality
54:21
and caliber of questions that came through.
54:23
It seems like this is front of mind for a lot of people, which
54:29
is reassuring to me, because I like to talk about how important
54:33
monitoring is all the time, but sometimes I wonder if anybody is
54:36
actually listening and it seems like there are people that are
54:40
out there listening and the explain ability part of this
54:43
Krishna has been really Useful to dive into what exactly it is
54:48
because it is a buzzword being thrown around.
54:51
It's a buzzword within a buzzword of ml Ops.
54:53
And so sometimes you have to really sift through a lot in
54:59
order to get there and figure out what it means and what it
55:03
actually does and why it's useful.
55:06
So let's see, Krishna Should I leave it to you?
55:11
Or you going to let me to please do.
55:13
Please do the honors now. You please select another I
55:17
think we got to go with Ron's man.
55:19
He quoted an article and everything he did some that was
55:23
some serious stuff that happened for Ron.
55:27
And then I really liked the question from I did like the
55:34
question from Depeche also about how to navigate politics and I
55:39
thought your answer Especially good because this is where
55:42
explain ability can shine, so maybe we can give it to Ron and
55:46
Depeche and everyone else. Thank you again.
55:50
We're going to be given away more Fiddler swag.
55:52
Don't worry, especially as much swag is they.
55:56
Let me I'm going to give it all away, but for Ron and Depeche,
56:01
please reach out to me or reach out to Krishna in the slack and
56:05
we will send you the swag bag and X again Krishna for doing
56:11
this man. This is super insightful.
56:13
Absolutely thank you so much Demetrius.
56:14
Yeah thing and it was a great great set of questions.
56:17
Thanks to the audience for joining in.
56:20
Yeah, thank you. Thank you all.
56:23
We will see you later. See you next time and if you
56:27
want to follow up the conversation with Krishna he's
56:30
in this slack. So ask away ask any questions.
56:33
We also have an explain ability AI slack channel.
56:38
So for more of these questions As in more of the news, that's
56:41
on this jump, in that stock channel, that is in our slack
56:44
workspace, because that is happen in places where all the
56:48
cool kids are like, a absolutely awesome.
56:52
Thank you so much. And yeah.
56:54
Nice day, everyone bye-bye. See you, everyone.