Latest / Agentic Conversations (formally mlops.community) / Approaches to Fairness and XAI // Murtuza Shergadwala // MLOps Podcast #142

Transcript

Approaches to Fairness and XAI // Murtuza Shergadwala // MLOps Podcast #142 · Agentic Conversations (formally mlops.community)

0:00
Hello everyone. My name is Mark Teixeira.
0:02
Voila, I'm a data scientist at Fiddler's AI.
0:05
I like to take my coffee with milk, it can be any milk, but it
0:11
needs to be milk. Welcome back everyone.
0:16
This is another edition of the MLA Ops Community podcast.
0:19
I am your host Demetrios. And today, I am joined by none
0:24
other than the one who shows up too much lately.
0:31
The incredible co-host, a b. So we talked today to my man
0:39
Moto, as I like to call him, because he told me that was his
0:42
nickname. And his real name is incredibly
0:46
difficult for me to pronounce, but Abby can pronounce it quite
0:49
clearly. It's Murtaza.
0:52
Exactly. And he is doing some incredible
0:55
work. When it comes to how humans and
1:00
Computers interact and what the experience is like the overall
1:05
experience. Abby.
1:06
What did you take away from this conversation?
1:08
I had so many good takeaways and I love chatting with him but I
1:11
want to hear yours first. Before I jump into it, the cool
1:14
highlight of it was the way he looks at the problems in ml
1:19
space. Back East for a very long time.
1:22
We've been looking at all of the problems from very much like an
1:25
engineering perspective where nobody was talking about the
1:28
design aspect of things, Building theme from a human
1:31
Centric perspective. I mean, one of the discussions
1:35
we had around others that was the highlight.
1:37
Yeah, at least for today. Oh, speaking about the alerts.
1:40
Let me pull on that thread a little bit more because I think
1:43
one of my favorite Parts was when he talked about how to gain
1:48
trust with the their customers, they see that their customers.
1:56
We'll set up alerts at zero just to see what they're getting
2:00
alerted on. And so, when someone starts
2:04
using Fiddler, they'll put their alert Threshold at zero just to
2:10
figure out what exactly it is. That is giving them alerts and
2:14
to make sure that they're getting the right alerts and
2:17
that way they can trust that Fiddler actually works.
2:20
So I thought that was a fascinating piece to dive into
2:23
again, I want to Echo what you Talked about the way he thinks
2:27
about designing things and this human Centric design and looking
2:31
at the whole picture is fascinating to me.
2:35
So, let's jump into this conversation.
2:37
A huge shout-out to Fiddler for sponsoring this episode.
2:43
If you are looking for anything, explain ability, you know, where
2:46
to go will leave a link to their website in the description
2:50
because it is worth checking out with guys like Moto at the helm,
2:55
all right. Right.
2:56
And don't forget, Abby loves me to tell everyone.
3:01
You should like, comment, and subscribe.
3:04
Give us a rating, and let us know what you think of these
3:08
episodes, because we love hearing feedback and especially
3:12
if it is the oddest kind that gives us a little bit of the
3:18
chills when we read it. So that's all I got.
3:22
Abby anything else has letters. He he didn't stick to the
3:25
script. And I love it.
3:28
All right, let's get into this conversation with Moto.
3:38
Oh do I have a? We're so excited to talk to you,
3:44
man. We want to just go right into
3:46
it. Well what were you going to ask
3:48
Abby, let's hear it. I want to hear your first
3:49
question. I'm excited.
3:51
So one of the interesting things particularly about your
3:55
background for me was you started mechanical engineering.
3:58
And I think a lot of you focus was towards human Centric.
4:01
Develop So weird. I don't, yeah, I come in and was
4:05
that, I sort of like a natural transition because you right
4:08
now, working at Fort Lee. And for me, it was like, uh, I
4:11
wonder how he survived moved into this direction.
4:14
Okay. Yeah, yeah.
4:16
That's that's I think one of the most interesting things,
4:19
obviously, which is connecting. The God says I started the my
4:22
undergrad, the back back in India and Idaho howdy, which was
4:25
focused, of course on mechanical engineering and I was always a
4:29
human Centric, like user-centric person, I I didn't know it at
4:34
that time or like, have the right words to talk about it but
4:37
mechanical engineering Pursuit started with Iron Man and
4:40
Transformers and things like that.
4:42
And I really wanted to get into that funnily enough.
4:45
I didn't enjoy the field of Robotics as much as I enjoy
4:48
people's interaction with the machines.
4:50
So even though, like I was expecting myself to be in this
4:54
robotics lab, I was more focused on, you know, getting to know
4:58
people and what they are doing in the different areas.
5:02
Has and connect with everyone. I had an internship in South
5:05
Korea. I had a study abroad in France.
5:08
I had gone to the US for an internship so I was literally
5:11
like away every year during my undergrad for all these
5:15
different internship opportunities.
5:18
And in fact, when I was at Purdue University, in in u.s.,
5:22
that's how I met my advisor and then I landed up pursuing a
5:26
direct Phu offer at Purdue the focus of my internships.
5:31
Also of course, adds That bad that I do in South Korea, that
5:35
was in the sustainable environment and Architecture
5:38
Lab. And they were very interested in
5:41
ubiquitous Computing, and that got me into design.
5:44
And then back at Purdue. It was the internship is focused
5:48
on crowdsourcing. That is leveraging ideas of the
5:51
crowd, but focused on engineering design.
5:54
And and this was primarily funded by NSF because NASA had
5:58
all these robotic design competitions and these
6:01
engineering competition Ins and they realize soon enough that
6:05
people were just trying to game the system, right?
6:07
There's Game Theory coming into the picture as soon as you put a
6:09
1 million dollar prize on designing, a robot every left,
6:14
right? And Center person is going to
6:15
try to participate in that the focus of my PhD then became how
6:19
do you design competitions in a manner that you can predict the
6:23
quality of outcomes that are going to be generated for these
6:27
domains specific areas? So that's how really the whole
6:31
in fact. Funny like I moved from the
6:33
crowd to a single human being. I realized that even on an
6:36
individual level, like cognition the mental processes and status
6:40
of that. Like we don't really understand
6:43
that as much so that was like really my condition, dude.
6:48
So there's some really interesting that I look at your
6:54
background and I automatically think like this idea of
6:58
decision-making behaviors of engineering designers Has in
7:02
design, contests. And basically what you were
7:05
doing is trying to figure out how people are making decisions
7:10
if I understand correctly. And I look at my life and I
7:15
think about how I can better make decisions, I guess the
7:18
question that I'm trying to formulate is coming back to the
7:22
individuality of each person in the way that they make
7:26
decisions. Have you found Tricks or tips on
7:33
how different people is it like, if I know my astrology sign,
7:37
then I can better understand the way that I make decisions or how
7:42
what were you able to find over your time studying this?
7:46
Because I'm just like incredibly fascinated by humans
7:50
decision-making. And the way that we feel about
7:52
that Yeah, that's that's an awesome thoughtful question
7:57
actually. And then when I started as this
7:59
like mechanical engineer, I really had a very different view
8:04
of like quote-unquote like social sciences and the way I
8:06
put it as close as Sciences is because I never thought of it as
8:09
a hard science because that's what was fake down to me as an
8:12
undergrad that engineering stem. These are the backgrounds to go
8:16
into and then I go into my PhD and my the first paper that my
8:21
advisor gives me to read. Is the works of Daniel Kahneman
8:25
and tversky focused on cognitive biases.
8:29
And I'm like, what the hell is all of this?
8:30
Like, I have, and how is this connected to mechanical?
8:33
Engineering like. And so, as I started reading a
8:37
lot about that and then want Neumann Morgenstern and Billy
8:40
the idea of like utility Theory and decision decision making and
8:44
all of that. But but yeah, I like talking
8:46
about like what I have learned. I mean, like, I there's just so
8:49
much like there are these, like, behavioral traits, of course,
8:52
there is I season so on, but but specifically with like engineers
8:57
and designers, what I realized is that it is more of an inverse
9:01
inference problem and that's where the work of Judea Pearl.
9:04
And all of that like became very, very important to me
9:06
because I was using Bayesian approaches and and my entire
9:10
work just started focusing on inverse inference, that is given
9:14
that we observe a bunch of decisions.
9:16
Can we infer? The latent variables, which we
9:19
can term it. As cognitive variables, which
9:21
are really like these Apps that people have over an entire State
9:26
space of actions or or that they can take.
9:30
What can we tell meaningfully about them by just looking at
9:33
their data and then adding some Theory with it.
9:37
So let me see if I understood that correctly because I think
9:40
there's something really interesting there when it comes
9:43
to what you're talking about. It's basically you're just
9:47
looking at the decisions someone makes and how you can infer and
9:53
gain Insight from the decisions. Oh yeah, that's right.
9:56
I can I can conquer dies this by giving an example.
9:59
So of course I've conducted control experiments and
10:01
behavioral experiments as a part of my PhD and since I had a
10:05
demographic that is mechanical engineering students really
10:08
available to participate. In these control experiments.
10:11
One of the settings that we had is that we really wanted to
10:15
understand when mechanical engineering students are
10:17
designing roller coaster, do they have a better understanding
10:21
of the physics of the problem as Back to F English majors are
10:25
designing, a roller coaster. And so we gave them this
10:28
sequential decision making problem because design is an
10:31
iterative process. We created this experiment where
10:35
we gave them this objective of maximizing enjoyment experience
10:39
by the Riders of this roller coaster.
10:41
And this is very true to real world, right?
10:42
Like you're given this objective, but there's no
10:44
mathematical quantification, do it.
10:46
So what do we do? Naturally, we conduct
10:48
experiments, right? We start ideating on these
10:51
different roller coasters and then we start experimenting with
10:54
them and experiments can also range from stress analysis of
10:58
these roller coasters to actually asking humans, who may
11:01
be potential Riders. Do you like the color or do you
11:03
like the shape, or whatever it is right there?
11:05
So many ways in which you can gain information.
11:07
So, I in my PhD, I kind of abstracted this process on a
11:11
higher level to create this framework in general of how
11:15
humans in any field. They're essentially any human is
11:19
a designer when they are trying to achieve an objective, whether
11:22
they are Sizing the objective or optimizing the objective, it
11:25
doesn't matter, but the higher level process is that you're
11:28
given an objective, you seek information, you are sampling
11:31
that space. You learn something from the
11:34
experiments that you're conducting, you update your
11:36
state of knowledge. And then you start either
11:39
refining your experiments or improve the space in which you
11:42
are and again, do the same process until you finally say
11:45
that. Okay, I'm happy with the
11:46
solution that I finally have. This is quite interesting but
11:51
I'm going to take the conversation of the a little bit
11:53
from taking the same foundations, where you say,
11:57
okay, let's understand how people are defining.
12:00
These latent variables to be decisions.
12:02
You also have had one of the papers, which is basically the
12:05
human Centric perspective, on Modern monitoring very said
12:09
basically the stakeholders desire customization of modern
12:12
monitoring systems. Now let's see where were talking
12:16
more so about how do we decide they are in?
12:18
For structure systems or any sort of ml Ops tools, how does
12:22
it apply over there and how much customization is actually
12:26
required and what's the process to go about it.
12:30
So, so actually these, these human-centric references when
12:34
they are making decisions, ultimately the inference problem
12:38
helps like if you can infer meaningfully something about it,
12:41
it aids in being able to design systems that are humans.
12:44
And and so now if you are designing, a model monitoring
12:47
system, ultimately I was using that system there.
12:50
This is being monitored by humans, who are looking at these
12:54
dashboards or they are, they are being alerted on maybe someday
12:58
to drift that's happening or maybe the system is behaving in
13:01
a weird. Now, different people respond to
13:04
information that is giving to them in different ways.
13:07
So some of these cognitive Norms that are there of, how do you
13:11
present information, is it visually important to them, or
13:15
is it numerically important to them, all of that?
13:18
Ultimately, Early influences, the decisions that they will
13:20
take off like, oh I do need to retrain my model or I need more
13:24
data or let me inform my bigger team about this model drifting
13:30
and how much drift is okay, for me, right?
13:33
Like these are all different decisions that people start
13:36
making based on what they see. So I feel like this, this idea
13:40
of preferences, they really can be embedded into these
13:43
monitoring systems, which are really the interface between the
13:45
human and the AI. When you are trying And to
13:49
create this ml Ops framework on top of your email system if
13:53
you're interesting and how does it work from like the designer
13:57
perspective of the people actually building these systems?
14:00
Because I mean it makes perfect sense from a user perspective
14:04
especially because a lot of people do often say and that's
14:06
one complaint that people have riches.
14:08
They are so many dashboards available printing.
14:10
Nobody actually ends up using dashboard people.
14:13
B hash board, and they just sit over there.
14:15
If I, let's see if we're trying to design a dashboard.
14:19
How do we take that kind of approach, human Centric,
14:21
approach to designing these kind of systems, and to be able to
14:25
make sure that we are actually providing that level of
14:28
customization, right? This is, this is a big problem,
14:31
and there are no solutions for it.
14:33
Of course, likes researchers, UI researchers like they, they
14:36
really do all of this work, right?
14:38
Like, interacting with the users.
14:40
Interestingly now, who is the user here becomes important?
14:43
So is it, are you, are you using, for example, Fiddler
14:48
software? That meant that is the company
14:49
that I work at. And let's say you're using that
14:52
software as a service. Are you a data scientist?
14:55
Who's going to be looking at the dashboard?
14:56
Or are you a c-level? Executives just wants a quick
15:00
summary of what's going on that change is the kind of features
15:04
that you provide in this dashboard system.
15:06
So knowing your audience, obviously matters.
15:09
What we have realized is that majority of the people, the
15:12
clients that we don't do it. It's funny that the decision
15:15
makers are at the sea level but then the people We're actually
15:19
aiding the decision are the data scientist and who ultimately
15:23
talk about, whether this thing is useful or not.
15:25
So catering to the needs of both is important.
15:29
Both from a business standpoint, as well as a utility standpoint
15:34
that dashboards and all those nitty-gritty details are going
15:37
to be seen by data scientist or Emily's.
15:39
In fact, we go to the level of separating out data scientist
15:43
and Emily's because Emily's might be incentivized in a
15:46
different way as compared to data scientists.
15:48
For their jobs, where Emily's care about production izing the
15:51
data, they gather about the model performance and so on,
15:55
whereas data scientist, might care about the quality of the
15:57
features bring in their domain expertise.
16:00
They want to be able to understand if any of the
16:03
features are drifting rather than performance and so catering
16:07
to both of these demographics becomes very important.
16:13
Our Emmanuel Mason machine learning engineer at stripe and
16:17
author of building, machine learning, powered applications.
16:20
And if you don't want your machine learning models to
16:22
explode, well, you should subscribe to this podcast. one
16:31
thing that I think about is something that I think you, you
16:34
spoke to little while back and it's how Explain ability and
16:41
just explainable AI as a movement and as something that
16:47
is important, in this field is very much changing as the field
16:53
changes and it is like evolving with the field and considering
16:58
how fast the field is evolving. Whether that be on the modeling
17:01
side or on the infrastructure side, if the explain ability is
17:07
trying to keep up, With that Evolution.
17:11
And so as you have your hat on of thinking about humans and
17:19
computers interacting together, how do you plan for that type of
17:25
thing? Like that happening.
17:28
Basically, my perspective since I'm a human Center design guy,
17:32
what what I have noticed is that these explanations are these
17:36
post, hoc explanations that we have for explaining.
17:39
Daddy or these attribution based approaches.
17:41
They are really model Centric. So they're trying to explain the
17:45
model, but at least they make an attempt to wear as we are trying
17:49
to present it in a very human Centric way.
17:52
And that's where I think the friction is coming in where?
17:56
Yes. Ultimately, on a higher level,
17:57
we are trying to explain the model so we want it to be the
18:01
human can build trust and understand it, but the
18:04
techniques that are there, they are faithful to the model and
18:08
the way. You're trying to present it is
18:11
that humans are able to put in their own cognitive biases, and
18:16
try to really bridge the gap between what they see versus
18:19
what they are thinking of. And, and the example for this is
18:23
like, for example, if you considered image explanations,
18:26
and you have a model that is doing a classification of
18:29
something like it, some bird, or not a bird, and we are trying to
18:33
understand, why does the model, look at a particular image and
18:37
claim it to be a bird. And we as humans have certain
18:41
features in our head that it should look at the beak or it
18:44
should look at the body and it should look at the wing and so
18:46
on. And hopefully it's not looking
18:48
at the sky or the clouds and labeling it to be a bird.
18:52
But when you look at these explanation post-op, explanation
18:56
approaches that are trying to show you, where the model in
19:00
general, is trying to focus on. It is just telling you that,
19:03
like, it's telling you, that these are the regions where the
19:06
model is focusing on. And we are the ones who are
19:09
Trying to impose our biases, that, okay?
19:12
If it is looking at this region, that means it has to be looking
19:15
at the beak, or it has to be looking at the eye, or it has to
19:19
be looking at the sky. That's something that we are
19:21
really adding on top of it. And I think that's where the big
19:24
gap lies with explain ability, that is training training.
19:28
The people who are actually looking at these interfaces and
19:31
letting them know, the limitation and assumptions of
19:34
these approaches is very crucial and we can't sell it as As a
19:39
tool that is for anyone and everyone like, it's not like
19:43
some person who has not designed the system or a data scientist
19:48
who does not understand post, hoc explanations and their
19:51
limitations can look at this image and look at the heat map
19:55
for example, and we convinced that, yeah, the model is paying
20:00
attention to these hottest regions, which means that okay,
20:03
it works, that's that. There is a, there's a big gap
20:06
there and I think it needs to be filled. the thing that I am
20:09
thinking about is you mentioned, how you can't sell it to just
20:13
anyone it has to be a someone who is interested in this but
20:18
also has a almost like a vested interest because it is important
20:23
for their job or whatever they're doing and I imagined
20:26
regulations come into play. And so for some people, it is
20:30
absolutely fundamental. So a who are these people that
20:36
you do sell to that fully understand and grasp the concept
20:42
and are on board. 100%, And then be the Gap.
20:48
How have you been trying to bridge that Gap?
20:51
I think of course, that is no on stacking practice that I'd like
20:55
the Baptist is evolving. As we are learning more and more
20:57
about how users are interacting with our system, but I feel like
21:02
it's almost like a user management cycle.
21:06
Just like there's a machine machine learning Model
21:09
Management cycle that you have all the way from.
21:12
You know, when you are training your data or do like production
21:14
IG, Saying and finally taking care of the life cycle of the
21:17
entire design of the ml model. It's almost like hand-holding
21:21
and and tutoring people as well in the entire life cycle, right?
21:26
And and that's where the human centered perspective comes in
21:29
that it's not just about calculating drift and
21:31
quantifying it but it's also about how you present it to do
21:35
the human and what kind of safeguards are in place.
21:39
So that you constantly D buys them because D biasing is not
21:43
just oh we on board. Them and we tell them something.
21:46
As I terms and conditions applied as a very little tiny
21:49
little statement at the bottom. So and then forget about it, it
21:52
has to be a continuous process where you keep letting them,
21:56
know that what you are looking at, is just the areas where the
22:01
model is focusing. This does not tell you anything
22:04
about the features, hey here is another tool that you could use
22:08
in conjunction with this, explain ability approach to even
22:12
develop that complete picture. Of what is your models?
22:16
Accuracy. What is your models performance
22:19
and, and it needs to be a constant B by saying and that
22:22
can happen through interfaces that can happen through training
22:25
that can happen through periodic cyclical reminders because we
22:30
like, when you, when you answered question, right?
22:32
Like who are the people who are 100% on board, I mean, the
22:35
people who are experts themselves, get confused, right?
22:38
Like and, and that's where the problem is.
22:41
It's not like these approaches right now, arcade of Go to all.
22:46
And, and it's important that you are in hands of a company that
22:51
cares about all these issues, right?
22:53
Like, yes, there are tools open source tools that are available,
22:56
but it's really like the responsibility lies with the
22:59
user at that point of time that here is a to will what you do
23:02
and decide with it is totally up to you.
23:05
If you do not have it managed or if you don't have like a DS team
23:09
at Fiddler's, for example, that can tries to establish
23:11
relationships with all these people who are trying to use
23:14
Software. You will not be able to have a
23:17
communication about it that hey, it's been 6 months, I have been
23:20
using this and then suddenly we realize as when we are talking
23:24
to our clients at all, actually you know what, we need a feature
23:26
in the software that lets clients know that this is not
23:30
what you need to conclude, or this is what you need to look at
23:33
and that improves the whole ux, the user experience.
23:36
But also ultimately improves trust.
23:39
So, it needs to be this very social, social technical
23:42
interaction. That is taking place.
23:44
Is even Within analogs. Very interesting and this does
23:49
bring me to the question that there are quite a lot of ml
23:54
monitoring tools out there. And one of the major Focus for
23:57
Riddler has been one is working on getting integrity.
24:00
And the other is more so working towards detecting liars,
24:05
alerting and dear draft such as all of those things, but outside
24:09
of that, the pipeline for an amount of is quite big.
24:13
So it goes from model, evaluation testing Hardware
24:16
metrics. And you can also work about cicp
24:19
pipelines. And so are you specifically
24:22
focused on just the data integrity and just the deer Pi,
24:26
which is let's let's help you out with this one and then you
24:29
use some other tool to work in conjunction on the ml
24:34
monitoring, which is the modern monitoring part.
24:38
No. So actually Fiddler provides
24:40
this entire stack of solutions and services to data Integrity,
24:43
but it's also model monitoring and it is also looking at both
24:48
production data as well as your training data and and you can
24:52
set up alerts on the ad. In fact, monitoring is not the
24:55
only solution that will is providing.
24:57
We also provide explain ability and fairness Solutions.
25:00
So it's like the three pillars as we like to talk about it and
25:04
so once you on board your ass, Actual model.
25:08
Then it allows Fiddler to even fiddle with the model and
25:13
predictions and so on if you're just focused on monitoring then
25:16
you don't really need to have your model on board.
25:19
You just need the input data as well as the production data and
25:23
we can just look at data, including your predictions as a,
25:26
maybe as a column, and see how it's drifting.
25:29
But if you really want explain ability as well, you have to
25:32
onboard your model and then we can start fiddling around with
25:37
it. Eating these on defectors and so
25:39
on and and understanding how your model is behaving in these
25:42
edge cases. So yeah.
25:44
All the three sort of pillars are they're very interesting as
25:48
in ml engineer myself. One of the most annoying things
25:51
to me is dealing with their loads because there are so many
25:54
alert people to get tired. There's the mental picture.
25:58
That's what happens sometimes. In everybody has to be
26:01
uncultured stations and all of that stuff alert fatigue, how do
26:04
you have work around that? And how do You sort of
26:08
understand because you have experience in it, c, i which is
26:11
which alerts are important and how do we think about these
26:14
alerting Solutions? Because this is a problem area,
26:16
at least 4 ml Engineers. 100% in fact, this my talk on in
26:21
Europe's was focused on that. So the human-centric
26:24
requirements and challenges in model monitoring focus a lot on
26:29
alert fatigue and cognitive overload and and how can you
26:33
create a system? And this is not just for machine
26:36
learning monitoring systems any Loading system, any monitoring
26:39
system in general has these issues but then how do you
26:42
contextualize it to ml, right? And if you can have this design
26:48
framework of okay, there is data, there is model and there
26:52
are model outcomes. And now if you can have this
26:55
higher level design framework, whether these different bins in
26:58
which you could start categorizing, the alerts, you
27:01
can at least on a higher level of abstraction, you let the user
27:04
know that. Hey, this is a data problem.
27:06
This is a Model problem. And usually in an organization,
27:10
one person is not responsible for the whole thing.
27:13
So, if you can't taste like like there are two things, right?
27:17
One is the alert system. And how many alerts have been
27:20
generated by? Then the other is who is
27:22
receiving those alerts? And so if there are ways in
27:25
which you can direct traffic to the right kind of people, for
27:30
the right kind of alerts that at least distributes the cognitive
27:33
load on an entire organization rather than one person.
27:37
So there are several approaches that you can use, but again it's
27:41
a hard problem. It's not like that are already
27:44
existing Solutions in there, but I do think that understanding
27:48
design Frameworks understanding what is it that people care
27:52
about in an ml system in its lifecycle which person is caring
27:56
about what, maybe even in the future Fiddler.
28:00
These are ideas that we throw around in the data science team
28:02
at Fiddler's, which is maybe we can design an inverse inference
28:06
system. Where we see, What the user is
28:08
interacting with to start categorizing the user into.
28:11
Okay, this is most likely a data scientist or this is most likely
28:15
a machine-learning engineer because they're interacting with
28:17
these different features which can automate it, an integrated
28:20
back into the alerts. You said there are a couple of
28:22
approaches. Can we serve, dig down into some
28:25
approaches and what are the limitations of each one?
28:29
Yeah. So of course, one of the things
28:31
is just starting with the user and doing user interviews and
28:34
research in terms of what kind of alerts you they really,
28:38
really want not giving them a decision fatigue, by giving them
28:41
a laundry list of all the different things that they can
28:44
do personalizing it to that particular client or reducing
28:48
that space of possible alerts which are the most important for
28:51
them, maybe even creating systems that are dynamic in
28:54
nature. Like as you interact more with
28:58
particular alert, They start showing earlier or do prioritize
29:02
them in that way. But, but again, there's a
29:05
trade-off there that just because a user is interacting
29:08
with a particular alert. Doesn't mean that another alert
29:11
does not really import. And so there, there needs to be
29:14
this domain knowledge infusion. Even in something as simple as
29:18
an alert, right diet, okay, what is the nature of the solute in
29:22
and of itself? Is this a very critical unload
29:25
or is it just a seasonal effect or something that Happened in
29:29
the day. So adding those attributes are
29:31
also are also important and then act.
29:34
As you can see, like it slowly starts becoming a heart problem
29:37
as to where you draw the trade-off.
29:39
But these are the kind of approaches that I can think of.
29:42
And this is this is also one of the easier purchase.
29:46
I see you around. That was as we defined
29:48
attributes for different kind of it out and using that we can
29:52
sort of train, a model to be able to identify those kind of
29:55
things and classify these kind of alerts into different
29:58
categories. Is.
29:59
But again, I also do understand that that would be almost a
30:03
naive approach because for anything which is an outlier, it
30:06
wouldn't be able to capture that one in.
30:08
Usually those are the most important ones, the heck so, so
30:12
that's interesting. So there hopefully, you're
30:14
designing an outlier detection system in and of itself that is
30:18
able to capture outliers and, and alerts.
30:21
If like, for example, if you're thinking about threshold based
30:24
alerts already, the threshold that you are setting for, A
30:29
particular alert, like for drift reduction, the value being less
30:32
sir above something for threshold bees alerts already.
30:36
It's pretty simple to detect all the outliers because, you know,
30:41
that anything above or below something would be an issue.
30:45
But it starts getting complicated when you're just
30:49
looking at data points in general and how far away they
30:52
are from these distribution shifts.
30:54
So there you don't want to act tribute or Is that allowed a
31:00
something that's not important or something that you can ignore
31:03
something, that's at the bottom of the list.
31:05
So, I think that one of the things that can be done is that
31:14
is, there is a chance for outlier detection with a
31:18
particular alert. Then you, you automatically,
31:23
like, make sure that you don't suppress that alert.
31:26
But then, in one of those days when there, Is an actual outlier
31:30
and the alert remains a breast, then don't then we run into
31:34
issues. Yeah.
31:35
So they'd becomes that design challenge, basically, that
31:38
making sure that you are not actively suppressing an alert.
31:42
When you know that there is a possibility that they can be
31:44
outlined is coming along. So, what I'm thinking about is
31:48
when you mentioned beforehand, what level of abstraction were
31:52
playing at and thinking about the other Type that you brought
31:57
to the surface. When you said okay, what about
32:01
the types of alerts that we actually are designing for?
32:04
So maybe you have some cool alerts that you've been seeing
32:08
and that you would suggest people to start looking at and
32:13
putting into practice because one thing was Abby's question
32:18
beforehand, how you mentioned? Well, maybe we can train a model
32:21
that can be predictive of these different.
32:26
But then it's hard to have those outliers and when you're talking
32:30
about what type of alerts that you put into practice, and there
32:36
are outlier alerts that you might want to have.
32:39
So what have you seen around that?
32:42
Because that just feels like a rabbit hole that we could go
32:45
down real deep, right? Oh yeah.
32:48
So I just give you one example Bev, though, you know, we, we
32:52
had this, we have this customer and and there's there.
32:56
Is a lot of data Integrity alerts coming up and, and it
33:00
made them realize that in the design process, there was a
33:04
little bit of modeling of null values.
33:07
And what does it mean for the values to be null?
33:10
And so Fiddler being Fiddler started, alerting the client,
33:15
the kesari, the customer about, hey, I'm seeing a bunch of null
33:19
values and that actually triggered a different thought
33:23
for the customer that actually, we Really put in a lot of
33:27
thought on what is considered null.
33:29
And so, it's almost like the alert system on though, it was
33:32
intended as a data Integrity violation.
33:35
It turned out to be a chain reaction of thoughts.
33:39
Design thoughts for your people have to come back and evaluate
33:42
as to, is this really a null value for us and if it is not,
33:48
then what is it? And that also triggered at
33:51
Fiddler's certain features, like, wavy in servicing, null
33:53
values. We call it fall back values.
33:56
Choose, which is the default values that our system is
33:59
adopting. And every bird, every
34:01
organization has a different approach to what is considered
34:04
fall backwards is what? What is null?
34:07
So, that's an interesting Chain of Thought that an alert, even
34:11
though we thought that it was intended for a particular thing,
34:14
decided in a different sort of alerting to the user and that
34:19
helped us actually even further characterize it, so, hit me with
34:24
some cool alerts, that you see, Or ways that you've seen alerts
34:27
being set up and creative alerts or all the different types of
34:33
alerts. If we're going to go down the
34:34
alert Rabbit Hole. So honestly like some of the,
34:39
some of the alerts that we see are pretty usual, right?
34:41
Like, which is the drift Matrix like so.
34:44
So we have this chance and Shannon Divergence and
34:47
population, stability index. And, and these value is being
34:50
above or below a particular threshold is very interesting.
34:54
In fact ass. Some of the customers just to
34:58
establish trust. What they do is that they set up
35:02
and alerting system at a zero threshold.
35:05
And they want a, the system to alert them for everything.
35:09
And that's how they start interacting with the system,
35:12
which is very cool. So, I like more of the
35:14
human-centric observations, right?
35:15
So, for me, it's not more about the alerts which are being used,
35:20
but how are they using it? And it's funny that alerts are
35:24
are actually a medium To establish, trust where initially
35:28
they're just like, okay, let me just test this, let me set a
35:30
zero threshold and let the alerts fire for the agree,
35:33
anything and everything that is happening.
35:35
And if the system does that then they start improving the
35:39
thresholds and seeing whether I'm getting a loaded before they
35:42
can be completely on board that okay?
35:44
I know that I'm going to be alerted on this very trusting
35:48
and this entire conversation I think this can go in so many
35:51
directions for one of the directions that does fascinate
35:54
me which is like Okay, but now, if you're designing sort of
35:58
monitoring solution that how do we further any sort of
36:02
dashboarding solution in a way that we are able to customize
36:06
things at different levels of the organization and is that
36:09
something which has to be done in a very client specific
36:13
manner, which is, let's personalized, XYZ for this kind
36:17
of client, or can it be done in an industry specific manner?
36:22
So for example, like the healthcare kind of copies to
36:24
have Miller challenges as a too late city commerce there that's
36:29
that's an awesome question II. Think that I don't think there's
36:33
going to be a one solution fits it all for sure.
36:36
I also think that that doesn't mean that we stop thinking on a
36:40
Frameworks perspective where, you know, especially from an
36:43
academic literature standpoint, like if there is research on,
36:47
how do you characterize a system and a subsystem or even for a
36:51
general industry and look at different types of hierarchies
36:55
and look at different during types of interactions of these
36:58
differing teens dad's actually influences the way product is
37:01
set up. I think that we're going to see
37:03
a reverse of that when it comes to is like Fiddler, where the
37:07
way we are setting up this Enterprise level software in,
37:10
the kind of people that are allowed to interact on this
37:14
actually will influence the kind of discussions that take place
37:18
between sea level execs and TS versus the marketing team versus
37:23
some sales team, that wants to look at The complete picture of
37:27
course, that's not really an answer.
37:28
But, you know, I do think that having these higher level of
37:31
abstractions like these, these visualizations of org charts or
37:35
system and subsystem Designs can Aid in the design of our
37:40
software as well. So we get very interested in
37:43
understanding those organizational hierarchies and
37:45
interactions because that will influence the way in an
37:49
Enterprise people communicate with each other.
37:51
Through this has been, so cool. Thank you so much for coming on
37:54
here and just Like exposing me to so many new ways of thought
38:01
that I had never dug into before.
38:04
And I find absolutely fascinating.
38:07
I really enjoy this whole conversation and looking at
38:12
human-centered design when it comes to doing things like model
38:16
monitoring or just interacting with ML Ops tools because that
38:21
is a whole different way of viewing the tools.
38:26
That I had ever experienced and I really like the thought and
38:31
the Deep care that you put into how someone is going to
38:35
experience it. So, thank you so much.
38:40
This is awesome, man. Now, this was this was super,
38:43
super fun. This is I feel like I want to do
38:45
more of these now we're here. Yeah faint.
38:52
Yeah, exactly. We're around.
38:54
So I've got my number. and if we can make it happen, This is
39:00
Skylar, I lead machine learning and health rhythms.
39:04
If you want to stay on top of everything happening in ml.
39:07
Ops, subscribe to this podcast now.
39:00
This is Skylar, I lead machine learning and health rhythms.
39:04
If you want to stay on top of everything happening in ml.
39:07
Ops, subscribe to this podcast now.