Latest / Agentic Conversations (formally mlops.community) / Building Trust Through Technology: Responsible AI in Practice // Allegra Guinan // #298

Transcript

Building Trust Through Technology: Responsible AI in Practice // Allegra Guinan // #298 · Agentic Conversations (formally mlops.community)

0:00
My name is Allegra Guinan, I am the Co founder and CTO of
0:04
Lumiara, the boutique advisory firm focused on responsible AI
0:09
and I take my coffee as a flat white with oat milk.
0:14
What is happening in good people of Earth?
0:17
I thoroughly enjoyed this conversation with Allegra, all
0:20
about the nuts and bolts and practicalities of responsible
0:26
AI. I've talked a lot over the years
0:29
about responsible AI because some of you may not know this.
0:33
I had a whole other podcast that was all about AI ethics,
0:36
responsible AI, all of those buzzwords that seem to be more
0:40
important every day we move forward.
0:43
This was pre ChatGPT that I was running that podcast, and
0:49
sometimes I felt like it was a lot of talk when we would get
0:54
into the topics of responsible AI or ethical AI.
0:58
And in this conversation with Allegra, it did not feel like
1:01
that at all. So let's stop wasting time and
1:04
jump in. All right, Let's talk about
1:12
responsible AI. That feels like a good place to
1:15
start. It is something I know you've
1:17
been diving into. And what do you mean when you
1:22
say responsible AI? Yeah, that's a good question
1:25
because there is no one definition of responsible AI.
1:29
When I think about it, it's an approach to the design, the
1:34
development, the deployment, the use and regulation of AI that
1:39
has a focus on a set of key principles that are related to
1:43
responsibility. And these range, like when
1:45
people list these principles, it can go from three to 10.
1:49
So when I think about it, there are definitely a few that come
1:51
to mind. So fairness is 1,
1:53
accountability, transparency, explain ability, privacy and
1:59
safety, reliability and robustness.
2:02
I know that's a lot, but they're all really important and there
2:05
are many others that are important as well.
2:07
But the the thought of responsible AI and the concept
2:09
is this approach that is holistic to every part of the
2:12
life cycle. It reminded me as you were
2:16
saying all these different parts of responsible AI, everything
2:22
ended in E. Like for basically any word in
2:26
the English Dictionary there ends with AY.
2:29
Yeah, that's in there. Those are all the principles.
2:31
Any word you can think of? So I would, but that's the whole
2:34
thing, right? There are so many different ways
2:36
to to think about it. And all these different
2:39
organizations are putting together frameworks around it.
2:42
NIST, for example, the National Institute of Science and
2:45
Technology, they have a, a risk framework for AI that's more
2:48
about transparency when they talk about it.
2:51
There's the International Association of Privacy
2:53
Professionals. They also have an approach and
2:56
then each company that's out there, any major company playing
2:59
in this field comes up with their own definition and
3:02
principles and, and puts it out there.
3:03
And that's what makes it really challenging as well to measure
3:06
to talk about, because nobody's really talking about the same
3:08
thing and it's up for interpretation of whoever's
3:12
talking about it at that time. Yeah, For me, it's a little bit
3:16
like the words that you're using can be highly subjective.
3:25
So even on the transparency, what exactly does transparency
3:28
mean? And you have it's OK, one
3:33
corporation is going to interpret transparency as one
3:36
thing, and then maybe a nonprofit is going to interpret
3:39
it as another thing. And so you're constantly getting
3:43
different shades of transparency, even though we're
3:46
saying, oh, we all agree that transparency is important.
3:51
Yeah. And not even everybody agrees
3:52
that it's important. And yeah, it does change.
3:55
And explain ability and transparency seem very closely
3:59
related. All of these things can really
4:01
be close. Transparency, when I think about
4:03
it, is having this visibility of what is going on when you're
4:08
building out a system so you can see how they thought through
4:12
design, you can see what kinds of decisions were made.
4:15
But then explain ability is really understanding, like a
4:18
human should be able to understand how we got to this
4:21
output, why those decisions were made.
4:24
And those things are different and quite difficult a lot of the
4:27
time as well. And so if a company decides
4:30
explainability is not their top priority, then obviously they're
4:33
going to build in a very different way and move in a very
4:35
different way. And I think that's because these
4:37
words are not really clearly defined and we're not aligned on
4:40
them. It also makes makes it hard as a
4:42
user to understand what questions you can be asking,
4:46
what you should be thinking about, or if you're looking into
4:49
buying an AI product or solution for your company, how do you
4:52
even figure out what you're supposed to be looking for?
4:54
And that's why it's really important for us to have these
4:56
kinds of conversations and open up the floor to how we define
4:59
them. OK.
5:01
So if I'm understanding that correctly, if your company does
5:04
not give you a clear guiding light or guiding principles on
5:09
how they see responsible AI, then the ripple effects you will
5:17
feel as you're going out there and you're trying to buy
5:19
something from the market and you don't know what questions
5:24
you need to be asking of that provider because you are not
5:27
quite clear on. But yeah, like we want something
5:30
responsible. Yeah, exactly.
5:31
And that's another way that I think about it quite
5:34
differently. It's not just a check box, it's
5:37
not just compliance. These are two different things.
5:40
So it's one thing to to have regulation, which obviously
5:42
there's more and more regulation coming out and so more companies
5:46
will be trying to just check off that box.
5:48
Yes, we are compliant even if it's bare minimum compliance.
5:52
But when we talk about responsible AI should really be
5:54
an organizational change, a cultural change that's coming
5:58
from leadership, but it's also the responsibility of everybody
6:02
in an organization. Even if you're just building
6:04
something on your own there, there should be this element of
6:07
understanding the intent of what you're building, why you're
6:10
putting something out there for people to use.
6:13
So I think a lot of the times the privacy team, legal team is
6:17
maybe doing the reviews of a vendor.
6:19
If you're working in an organization or you have
6:22
engineering and they're suddenly the ones building everything.
6:25
Especially in this space where things are moving so fast and
6:28
product managers or whoever are like, yeah, just go build this
6:32
as fast as you can put it out there.
6:33
And leaders want to see that. So suddenly it's an engineer's
6:36
responsibility to understand this term and to put it out
6:39
there. And if you don't have alignment
6:41
on that within an organization or within a region, whatever it
6:45
is, as big as the scale you want to be or small, then you're not
6:48
going to end up with a result that everybody agrees is fair
6:51
and ethical and all these different things.
6:56
Do you feel like this is something that is more pertinent
6:59
for larger organizations? I don't think so, because there
7:04
are a ton of small organizations that are building things that
7:07
could end up getting a ton of use and reach and being snowball
7:10
effect. They put something out there and
7:12
somebody else builds on top of that or whatever.
7:14
I think it really should just be a part of the culture of
7:18
development. I don't think it really matters
7:20
what scale the organization is and even if it's just an
7:23
individual, like I said, I think having that understanding of
7:27
what the intent is, why you're actually building something.
7:30
Do you have a positive intent? Is it negative?
7:32
Are you keeping track of the decisions you made even for
7:35
yourself? Do you understand why the system
7:37
that you're building is making the decisions?
7:40
And if you don't, I think that's really a time to step back and
7:43
and reflect on why you're doing it, which is hard because people
7:46
are super excited and they want to build really fast, which
7:48
makes sense. But I think it's easy to get
7:51
carried away and then end up somewhere that is really far
7:53
from what you originally intended.
7:56
And do you have a set of questions that you tend to
8:02
encourage people to ask, like just what you were mentioning,
8:06
What's the intent of it? Why are you building it?
8:08
How are decisions being made? That feels like a great start.
8:12
It also feels like something that anyone that is involved in
8:15
the process should be doing. Now, whether or not they do it
8:19
is another question potentially. That's because there's a lot of
8:23
friction in answering those questions.
8:26
Or I just answer it in my head as of coding my function, and
8:31
I'm like, yeah, of course I know why I'm doing this, but that
8:34
doesn't necessarily codify it for the whole organization.
8:38
So have you seen best practices on questions to be asking and
8:44
then how to make those questions discoverable for the whole
8:49
organization? Yeah, I think actually one step
8:52
before that is who is in the room when you're coming up with
8:56
the kinds of questions to ask when we're, and I'll just back
9:00
up for a second. So I am the Co founder of a
9:03
advisory firm that's focused on responsible AI.
9:06
And so we are helping organizations make better
9:08
decisions around their AI, the way that they're building it,
9:11
the way they're using it. And so one of the first things
9:13
we do is when we're starting to speak with with leaders with an
9:16
organization, understanding who's in the room around
9:19
decision making. And one of our core values is
9:22
perspective density. And when we think about that
9:25
term, when you think about density in the regular sense,
9:29
like how much you can fit into a certain area, perspective
9:32
density is how many different perspectives can you have in a
9:34
given space, in a given room. So if you enter a room and
9:39
you're starting to ask these kinds of questions that you just
9:41
brought up, who do you have there in the space?
9:44
And if you only have a very small group of people that are
9:47
from the same side of the organization or have the same
9:49
background, then the kinds of questions you ask will not end
9:53
up in a very fair place or responsible place.
9:57
So I think before you even come up with that, it should be, do
10:00
we have everybody in the room that we think is important to be
10:04
here? Probably not at 1st and that's
10:06
fine. So understanding where the gaps
10:08
are and how can you bring in more views so that you can come
10:11
up with the set of questions that will lead you to the kinds
10:13
of answers that you need. I like this perspective density
10:19
and recognizing that you probably do not have all the
10:24
right voices in the room. Or maybe it's that you need to
10:30
socialize some type of a piece of writing to a broader group of
10:35
folks to get that perspective density.
10:39
Yeah, definitely. We only know what we know.
10:42
We only have our own lived experiences and that can be
10:45
really diverse within ourselves. And we could have had a ton of
10:47
different lifetimes that we've lived in various places or
10:51
whatever, but it's still just ours.
10:53
And so as you're starting to work on some sort of product
10:57
solution that will have far reaching impact and potentially
11:00
be more out of your control than something previously that we've
11:04
had in this area in technology, how can you bring in more views
11:09
to make sure that you're building something that can be
11:13
used in a fairway by a lot of different people?
11:17
So my instinct instantly is all I got to bring in more like
11:22
stakeholders. This is going to make things so
11:24
much slower or it's going to somebody, of course, is going to
11:28
want to be like a dog peeing on a fire hydrant.
11:31
And maybe what they say doesn't actually add much value, but
11:35
they have to feel like they're contributing in one way or
11:38
another. I, I think we've all worked with
11:40
that person before. How do you keep the conversation
11:47
productive while bringing in those different stakeholders, I
11:51
think. There are a couple different
11:52
ways. So 1 is this cultural shift as
11:55
well. So I think it shouldn't feel
11:57
like a burden. It's really a change of mindset
12:01
of how you build. If you are under a lot of
12:04
pressure to build really quickly, for example.
12:06
And then you're also given these roadblocks, what might feel like
12:09
roadblocks, where you have to have more conversations, you
12:12
need to create more artifacts, whatever it is.
12:16
Then that will go against what you feel like needs to be the
12:18
top priority because that's what you've been told previously.
12:21
So I do think there has to be an understanding if you're working
12:25
in an organization or on a team, where it's OK to take more time
12:30
to do things correctly. And I think you can apply that
12:33
to anything, right? Even before responsible AI, you
12:37
want to work on foundational things, you want to work on your
12:40
tech that you can't just keep building faster and faster and
12:43
not worry about all these other things.
12:45
So it should be that same mindset of you're just taking a
12:47
little bit more time to build things out correctly so that
12:50
they are more reliable, scalable, robust, and then you
12:53
can build faster after that. If you have these set of
12:55
principles upfront and you know what you're supposed to be
12:58
doing, the kinds of decisions you should be making, what to
13:01
consider, then you can be faster later.
13:04
So I think giving it a bit more time at the beginning is, is one
13:07
thing and having this shift of mindset.
13:09
The other is creating technical requirements that are really
13:13
easy to work around rather than having just a bunch of policies,
13:18
a lot of back and forth with conversation, which I do think
13:21
needs to happen. But maybe it doesn't need to be
13:23
the engineers all of the time involved.
13:25
Although I do think engineers need to be present a lot more in
13:28
business conversations. But for example, the EU AI AC
13:32
technical requirements that the guys from Lattice Flow build out
13:37
like Comply AI or however you say it, I think that was a
13:40
really good example of how to make something really tactical
13:44
and not be this massive regulation that you're not
13:48
really sure how to break down and use it everyday.
13:51
So I think if we can do more of that and go more in this
13:53
direction of how you just build in checks, have metrics that you
13:56
all align on that are related to these concepts and they're just
13:59
part of your workflow, then I think it makes it a lot less
14:02
daunting to to have as an engineer.
14:06
How have you seen business requirements being translated
14:11
into those technical requirements or law documents
14:16
being translated into technical documents, such as what happened
14:21
with Comply? Yeah, I think we're not seeing a
14:25
ton of it, to be honest. That was the first technical
14:28
breakdown of the UAI Act, which is obviously a very new piece of
14:32
regulation. It's not something that I've
14:34
seen used a lot and not that specific use case, but in
14:38
general technical requirements that are directly related to
14:42
something like that, some piece of legislation or whatever it
14:46
may be. I think if you've been working
14:49
and and development for any period of time, you do see this
14:53
lack of connection between what people want originally and then
14:56
what ends up being built. I've been a product manager,
14:59
technical program manager, like I very clearly see that
15:02
disconnect all the time. So we're not even good at doing
15:05
it to begin with, let alone in this kind of new area.
15:09
So I think there's a lot of opportunity there for sure.
15:12
And it's definitely something I want to see a lot more of.
15:14
And that's also why it's important to have engineers in
15:16
the room because then you get this perspective as well.
15:18
This is what would be easy for me to understand if it meets
15:22
your needs. These are the kinds of questions
15:24
that would come up for me when I'm building to know if I'm
15:27
doing this in the way that I should be or that you want it to
15:31
be. So I think having those
15:33
conversations and figuring out how to format these requirements
15:38
into technical requirements is something that should be
15:40
happening more. We do have metrics obviously
15:42
that are available out there. I don't think they're as widely
15:46
used as some performance metrics, but there are some for
15:49
bias and toxicity. You still have to define what
15:52
your threshold is. So again, there has to be a
15:54
conversation with whoever's building, with the organization
15:57
of what that means for you. And what is the line where you
15:59
say this is not appropriate, We will not move forward.
16:02
But they exist out there. And I'm hoping that we see more
16:04
and that they become more socialized and regularly used
16:07
in, in everything, because right now I'm seeing a lot of, yes,
16:12
the context was accurate. Yes, there was low latency, but
16:15
these are performance things. It's not actually saying if this
16:19
was harmful in the end or even if it was useful necessarily to
16:24
to whoever was on the other end. So I think changing a bit how we
16:28
think about metrics, really making that a core part of the
16:31
planning process upfront and incorporating more of the
16:34
concepts of responsible AI into those metrics so we can measure
16:38
them. It almost feels like some of
16:41
this stuff is playing on a different level than the actual
16:45
like technical part of it, right?
16:48
Like you were talking about performance metrics there.
16:52
Is it harmful or is it useful is more of a product metric In my
16:57
eyes. It feels like the those are
17:01
almost bigger questions to be asking and hopefully you're
17:04
asking them earlier on in the conversation.
17:07
Then when you've got the AI out deployed and then you recognize
17:13
actually this doesn't, it's not really that valuable, which
17:16
we've all seen happen plenty of times just because someone wants
17:20
to be using AI or at least say they're using AI.
17:26
Yeah, they are different, definitely.
17:27
I think because right now we're seeing a lot of engineers
17:32
driving development because they're the ones who are the
17:35
most literate in the space. Companies, organizations are
17:39
realizing that they have people on their team that are able to
17:42
build what they think is necessary.
17:45
And so a lot is being put on the engineers.
17:48
And so I think naturally it leans more performance because
17:52
that's what you're thinking about as you're building.
17:55
You're like, OK, yes, this is exactly what I wanted.
17:57
It's not, but I expected what whatever.
17:59
However you're defining it for yourself.
18:01
And so it's not really the same mindset as with product
18:05
management in general or building out a product where you
18:08
have this whole planning phase, you define your success metrics,
18:12
those are impact ones and product related and performance
18:16
and you have this whole thing, you have a ton of artifacts.
18:18
It goes through a process like none of that is really happening
18:21
right now. People are not thinking about AI
18:23
as a product. So much is put on engineers to
18:26
just build and and put it out there.
18:28
And so that's what I'm calling out, that there's a lot of focus
18:30
on performance metrics just naturally because of how things
18:33
are evolving. And I think there needs to be a
18:35
shift where we are considering other elements, other concepts
18:41
and bringing that into the conversation as well.
18:43
And when I did this, the ML OPS meet up on responsible AI on the
18:47
tactical side, on the engineering side.
18:49
And I was asking like, if a company decides they care about
18:54
responsible AI and they put out all these things and then they
18:57
asked the engineers to like work non-stop, put things out all the
19:01
time, is that really responsible?
19:03
They're impacting their workforce.
19:04
They're not giving their engineers the tools they need.
19:07
They're not bringing them into the conversation.
19:09
They're saying they're doing responsible AI.
19:10
But I think there's so much pressure on engineers right now.
19:13
And they're really the only ones that are driving a lot of the
19:16
conversations that I'm seeing forward on the tactical side.
19:19
There's a lot of theoretical conversations and regulation and
19:23
things like that, but at the end of the day, I'm not seeing a ton
19:25
of connection between that and then the engineers.
19:29
Which I think is a lot of the backlash that the responsible AI
19:34
movement in general gets is that it is a lot of talk, but when
19:40
the rubber hits the pavement, what do you actually do?
19:43
That's nice that you're calling it out and saying, look,
19:46
engineers are being forced to create things.
19:49
And then you have other folks that are saying, yeah, we want
19:54
it responsibly. They're not clearly defining
19:57
what responsible means, and they're not really giving the
20:00
engineers the space to create the responsible AI that they're
20:07
looking for. Yeah.
20:08
It's exactly that, and the literacy part is so big here for
20:13
responsibility. There is a lot of technical
20:16
literacy on the engineering side of things, and I think there can
20:20
be areas of literacy related to AI in teams like privacy who
20:24
have to be really aware of what's going on in terms of
20:27
regulation. But then everywhere else, even
20:30
at like C-Suite level, maybe even especially depending where
20:34
you are, you're not seeing a ton of time put in to really
20:38
understand what they're asking for and to understand the
20:42
capabilities, limitations, who they have on their team right
20:45
now, who they don't have in order to make the kinds of
20:49
decisions and move the organization forward in the way
20:52
that they have in mind. I think there's a, a huge onus
20:55
on the other teams outside of engineering to be part of this
20:58
conversation to to create these definitions, to create that
21:02
culture and then put a framework into place that makes sense for
21:06
everybody. And not just put it onto one
21:07
team or another and say, go figure this out.
21:10
I want something transparent, but I won't tell you what that
21:12
means. Good luck.
21:14
That's where we are right now in, in the the conversations I'm
21:17
seeing. And I think there needs to be a
21:19
huge shift. It's good.
21:20
It's like the Charlie Munger quote, tell me where I'm going
21:23
to die so I can never go there. You're telling us everything
21:27
that we shouldn't be doing. So hopefully we do the opposite.
21:30
And it begs the question, what are teams doing that are that
21:35
you feel are doing this well? What are some of the things that
21:39
they're doing? Basically the opposite of what
21:41
you're talking about it, but maybe you've seen certain ways
21:44
that certain strategies even that have proven to be very
21:50
effective. Yeah, I think having a strategy
21:54
to begin with is quite effective, not just going out
21:57
there and trying to do way too much without a plan.
22:02
Maybe I just love a plan, but I do think it's helpful to have
22:05
some sense of where you're going and why you're going there.
22:08
Starting small I think is quite helpful in an area where you
22:12
have a lot of a lot of people who are engaged and
22:15
understanding and who are going to work together on on building
22:19
something that's impactful in a responsible way.
22:22
Try to just go after one area and then build from there and be
22:27
like, OK, this worked. We proved our value.
22:29
This is not any new kind of information that I'm sharing,
22:32
but we're still not necessarily seeing that.
22:34
I think the one of the defaults modes is like, just put AI in
22:38
everything. And that also could happen
22:41
because maybe you have a lot of applications that you use and
22:44
suddenly they all have AI in them.
22:47
And so it happened to you. But I think just starting small
22:51
and focus and building up and seeing where you actually have
22:54
value being delivered is one thing.
22:57
Another is shared learning. So I think organizations that
23:01
are finding out who the people are that are on the team already
23:05
who are super excited about things.
23:07
They have started a channel, Slack channel, whatever it is.
23:10
They're starting to host weekly sessions.
23:12
They're getting really excited. Those people have a lot of
23:16
knowledge and are really dialed in and that can be really
23:19
helpful to spread literacy across an organization to be if
23:22
once you come up with whatever your principles are, whatever
23:25
responsible AI means to you, those people can be so helpful
23:28
in getting that out to everybody else.
23:30
So I think that's under leveraged right now as well.
23:33
But companies that are that have that and organizations that have
23:36
that I think do a lot better in terms of moving things along and
23:40
in the way that they have in mind.
23:41
And I think this is just across the board something that isn't
23:44
utilized enough or celebrated enough really are the people who
23:47
are champions for new things, who are naturally starting the
23:51
kind of conversations that you would want and doesn't have to
23:54
be forced. There will be people who will
23:57
drive that forward. Yeah, the evangelists that are
24:00
doing that, it's such a great call out.
24:02
When you talk about a plan, what is a successful plan?
24:09
What does that look like? And what about, on the other
24:14
hand, when we've already got something going and now we're
24:19
trying to retroactively go back and make it responsible in a
24:23
way? Yeah, it obviously depends on
24:26
what success means to you, what that plan ends up looking like.
24:30
So maybe that's just the most important part is have you
24:33
define what success is. Is it overall adoption?
24:36
Is it increased literacy across your team?
24:39
Is it to be called out as one of the the innovators in this
24:43
space? Do you want to build something
24:44
that's never been built before? What are you trying to do here
24:47
and do you understand what that is?
24:49
So that's the very first thing, not again, not just saying we
24:52
want AI and then that's it. That's not enough of a plan.
24:56
So I don't think you need to know every single detail.
24:58
It doesn't need to be prescriptive in that way of
25:00
these are the exact solutions that we're trying to put out
25:03
there. But having an understanding of
25:05
what success would mean for you, I think is the best place to
25:08
start. So obviously you can make
25:10
updates to things like not once you put something out there
25:13
doesn't need to be the only version of that ever exists.
25:15
I hope. Think of some sort of iteration
25:17
process in mind. This is actually something that
25:20
we do as well. So a lot of people have PO CS
25:23
out there that are failing. They didn't deliver the ROI that
25:26
they expected from so many different reasons because they
25:29
didn't define success, because they didn't roll it out in a way
25:32
that makes sense to their workforce or to their customers.
25:35
They didn't come up with the right considerations, whatever
25:37
it may be. So I actually think it's a
25:38
really good exercise to have something that you already
25:40
pushed out that isn't necessarily delivering what you
25:44
imagine. And hopefully you didn't choose
25:46
something that your entire business deends on.
25:48
And it was quite small and controlled.
25:51
So you can go back and say, OK, if we could have done this
25:53
differently, what's not going super well?
25:56
Where are we seeing potentially outputs that don't align with
26:00
our values or our vision or whatever it may be?
26:03
And let's make changes around those specific things.
26:05
And now we know we need to have experimentation in place or we
26:08
need to have some sort of framework for evaluation in
26:11
place that we didn't have before.
26:13
So I think it can be really helpful actually if that's the
26:15
position that you're in, because you'll very clearly see and very
26:18
quickly see where those gaps are and where you're starting to to
26:23
fill on things that are quite important.
26:25
Yeah, that is such a great call to leverage the failures as a
26:30
learning and then recognize like all right, next time we do this,
26:34
how are we going to do it differently?
26:35
Yeah. This concept of infallibility is
26:39
so interesting. I don't know if you are reading
26:41
or did read Harare's Nexus. So he is the author that wrote
26:46
Sapiens, which was a book that a lot of people enjoyed, and Nexus
26:50
is the the latest one. And so he's talking about
26:53
information as a concept and he ends up talking about AI as
26:57
well. But there's this whole concept
26:59
of infallibility. And if you think that you are
27:02
infallible and you don't plan for anything around failure, you
27:06
will fail. Obviously, it's inevitable, but
27:09
you'll have nothing to fall back on and you will ultimately be
27:12
destroyed. And this he talks about it in in
27:14
terms of really large institutions and organizations
27:17
like the church, for example, where they consider themselves
27:20
infallible. And this applies to technology,
27:23
it applies to small systems, big systems.
27:26
If you don't plan that something will go wrong and that you will
27:30
have to iterate and you think everything will be perfect and
27:33
this is the be all end all perfect solution.
27:36
Ultimately it won't work, it will fail and you'll have
27:39
nothing after that. So I think it's a really
27:42
interesting concept that we should think a lot about when it
27:45
comes to technology when we're building.
27:47
This very closely translates into robustness as well, which
27:52
falls into one of these principles that's quite
27:53
important for responsible AI. So different from reliability,
27:58
robustness is OK, there's a path.
28:01
I'm going to create more paths because if this one doesn't
28:04
work, then I will be able to handle any sort of situation
28:07
that comes my way as pivot and and be flexible, adaptable.
28:12
That's also super important for anything that you're putting out
28:15
there now. So when you're building, you
28:18
should plan to iterate. It should not be the last
28:20
version that you're ever putting out there because we're going to
28:23
make mistakes. We're going to miss something,
28:25
but you're not going to have thought of everything because
28:27
you didn't have everybody in the room because you were moving
28:29
quickly. Just naturally it's going to
28:31
happen. So accepting that first of all
28:34
and building in a process to to handle it when it happens.
28:38
How does that look like with an actual product?
28:41
Let's take an example. Maybe you've seen some AI
28:45
products that have been put out there and what is robustness
28:49
equal for those products it besides just like iterating on
28:54
the product? I'm not.
28:55
Sure. If I've seen a ton of really
28:58
good examples. I'm definitely seeing examples
29:00
when things are failing and I'm seeing also it's being rolled
29:04
back. I'm sure you are as well.
29:06
We've all seen these stories that have come out this entire
29:10
year of major companies, large language models, doing crazy
29:14
things and putting out text that is completely crazy.
29:19
And then things ultimately get rolled back.
29:20
And I'm sure then there are changes made and apologies made
29:24
and then things are fixed. It's hard to tell when it's not
29:28
that. I think it would probably be
29:30
more subtle and wouldn't necessarily make the news for a
29:33
lot of people, and especially if you don't have the biggest
29:35
organization. But I think the most important
29:37
thing is to plan for that. And it's fine to to iterate.
29:41
This is also where transparency comes into play, right?
29:43
So if you have something and you're working with AI and you
29:46
put it out there into the world, having people understand that
29:49
they are interacting with AI is really helpful and I think
29:54
highlights responsibility as well.
29:56
Like these answers may be incorrect.
29:58
They are based on data from this.
30:02
This data set doesn't need to be specific.
30:04
But up until this point in time, we have these limitations.
30:07
I think calling that out more than we see already, which is
30:10
quite limited I think at the moment also because a lot of
30:13
people don't know they're building on foundational models
30:16
that they have really limited visibility into.
30:19
But I think being a lot more explicit about what the gaps
30:22
that we do know so that users understand that, OK, this may
30:27
not be perfect, I'm going to adjust my expectations a bit.
30:30
So it's fine if this ends up being rolled back and iterated
30:33
at everybody's still OK in this understanding that was going to
30:37
happen. So I think it's just that
30:38
preparing and and being adaptable and understanding that
30:41
you're not going to get it right the first time.
30:44
Yeah, it's funny how in the EUI think one of the things that
30:48
they put in the EU Act is that if it's AI, it has to explicitly
30:52
say so. And talking with a few folks who
30:56
have built voice agents in the US, they say how redundant that
31:02
is. Because I asked specifically, do
31:04
you have the voice agent identify itself as AI when it's
31:09
talking on the phone with people?
31:11
They said no, because people figure it out pretty quickly.
31:15
They understand there's something not quite right with
31:19
this. And so to say it, it doesn't
31:23
really accomplish much. And since people already know,
31:28
they know something's up and they know it's not a human so
31:31
that they can guess what it is. And you don't get, you don't get
31:36
that far by saying I'm an AI voice agent.
31:39
OK, let's get into our conversation.
31:41
All that happens is going back to the metrics.
31:44
If you say that as the first thing when you identify yourself
31:48
as a voice agent, people just hang up.
31:50
And that doesn't get you to have people have this experience with
31:55
your voice agent, right? I think that is going to be a
32:00
hard one to hang on to because obviously these agents will get
32:04
better and better, like quality is going to be a lot higher.
32:07
There will be a lot more personalization as well.
32:10
So like a bot will know how to talk to a particular person to
32:15
really resonate with them. And so I, I don't know how long
32:18
that that argument will hold up. But I also think this is where
32:23
the literacy part comes back into play.
32:25
So I think the general public still has quite a a
32:28
misunderstanding or there's a large gap between understanding
32:31
the the limitations of the systems or what they do, how
32:35
they work. I think a lot of it is just OK,
32:38
yeah, I know ChatGPT exists or yes, that sounds like a bot, but
32:43
what does that actually mean and what can I trust?
32:46
What should I trust this to be 100% accurate?
32:50
We talk about hallucinations in general.
32:53
I know you talk about it as it's on your merch.
32:56
It's been less actually this year, I think like couple years
32:59
ago, it was more the topic, but people don't even understand
33:02
that concept. Like the general public,
33:05
they're, we're in such a bubble, especially in engineering, it's
33:08
an even smaller bubble within this entire thing.
33:10
So a random person isn't really going to understand what those
33:14
lines are. And I don't think it's fair to
33:16
make that assumption. And again, when you're building,
33:19
when you're designing, this is how we're going to deploy this.
33:22
This is what the experience will be like.
33:24
If you don't have an understanding of what your
33:26
general user thinks about or how they interpret that, then you're
33:31
going to make a decision that is not aligned with reality.
33:34
And maybe it's your assumption that they would hang up the
33:36
phone if they're talking to a bot.
33:38
Do you know that? Have you actually surveyed your
33:41
user, baby? Yeah.
33:42
Do you understand how they feel about AI?
33:44
Maybe you'll make different decisions if you understand that
33:47
they're skeptical or if you hear from them that it's because they
33:51
want a different kind of experience.
33:53
Maybe you can build that instead of just building something based
33:57
on an assumption and then an assumption on how somebody would
34:01
respond to that, etcetera. I think it that's part of the
34:03
problem, right? We're just making a ton of
34:05
assumptions and not actually understanding who's on the other
34:09
side of things. Yeah, because I instinctively
34:12
think, and this is my assumption, if they're
34:15
interacting with a voice agent, they just want to get done what
34:18
they called in for. And if it is a human that helps
34:23
me get what I'm calling for, great.
34:25
If it's a voice agent that helps me get it, great.
34:28
If it's me having a bang, a few numbers on the telephone and
34:33
it's robotic, completely great. Just as long as get it done and
34:38
get me off of the phone as fast as possible, yeah?
34:41
If in this example where somebody has built a voice
34:44
agent, if they really understand their user base and the main
34:48
thing that people care about is just getting the thing done as
34:50
fast as possible and they have filled out our survey or done.
34:54
The team did their due diligence and they really understand.
34:56
People don't care if they're talking to a bot or not, then it
34:59
doesn't matter. If you make a transparent and
35:01
announce that you're talking to a bot, maybe the bot can say I
35:05
am an AI agent but I will get this done in one minute
35:09
guaranteed faster. Than the humans, Yeah, faster.
35:12
Than a human, whatever it is like you can make it fit what
35:15
your user base wants. I don't think that's a excuse to
35:19
not follow this kind of regulation or to understand the
35:23
purpose of it. I think it's a bit of a
35:26
deflection. Yeah, it makes sense and it is
35:30
fascinating to think about how certain areas of the world are
35:37
really putting guardrails on from the beginning, like with
35:40
the EU act making it mandatory for AI to call out that it is
35:47
AI. And then other parts of the
35:49
world in the US, Yeah, do whatever you want, Wild West on
35:53
this and whatever is best for or it's almost like saying the free
36:00
market will decide in a way. Yeah, people talk a lot about
36:06
these differences and I know that there's a a perception and,
36:10
and might be real as well, like regulation is getting in the way
36:14
of innovation and things moving quickly.
36:16
And the EU obviously has a very different approach to this than
36:19
the US, and the rate of things moving is reflecting that, of
36:23
course. But yeah, I still support it.
36:25
I still support having this idea of understanding intent, making
36:29
sure that it's not harmful, taking steps towards creating
36:33
the tools that you need in order to put things into place so that
36:36
you can actually build things in a way that doesn't put humanity
36:40
at risk. That's just my stance on it.
36:43
I know not everybody feels that way, but I think it's important
36:47
like this. The mission of Lumiere, this
36:51
company that I am the Co founder of is a future equipped for
36:55
humanity. So rather than adjusting the way
36:59
that we are to fit technology as it's coming at us, this is an
37:02
opportunity for us to shape technology to support and
37:06
protect everything that is human and that we want to maintain.
37:09
And I think that's that shift in perspective is where you also
37:13
have this where if you're thinking about responsible AI as
37:17
something that's getting in the way of advancing technology as
37:22
fast as possible, are you still considering the human aspect of
37:26
it and what you will have left of humanity once you've reached
37:30
this goal of getting your AI system out there as quickly as
37:33
possible and expanding to X amount of people?
37:36
What have you actually accomplished for humanity if
37:40
you've completely wiped us off the map or whatever it is,
37:43
whatever the potential negative outcome is once it's out there.
37:46
So that's where I think the difference is.
37:49
And I think obviously not everybody agrees and there will
37:51
be some people who don't think the future needs to be equipped
37:54
for humanity, but it's something that I really believe in by.
37:57
Saying that, you are stating there are certain things about
38:04
our human experience that we want to preserve or we want to
38:09
make sure, as you mentioned, the future is keeping those intact.
38:16
What are those things in in your eyes like the the human aspect
38:20
of it? Yeah.
38:22
One thing that we talk about a lot is friction, which I know is
38:25
against everything that people want to hear in the tech space,
38:29
but I really think it is something that makes us human.
38:31
And so we're heading into this direction of hyper personalized.
38:35
Everybody is seeing something that is so tailored to them and
38:41
it's as fast as possible. They have to jump through 0
38:44
hoops to get there. You never have to really think
38:48
anymore and you're never in a challenging situation.
38:51
And I think that is something that makes us really human,
38:53
having to be in an uncomfortable place, take a step, be in a
38:59
position that everything takes a bit longer.
39:01
If you've been in a different country and you don't speak the
39:02
language and you don't have a translate app or whatever, it is
39:06
like readily available to you, being able to read the signs or
39:10
get by or ask a stranger something and figure it out.
39:13
I think that is such a powerful and life shaping experience.
39:17
And it doesn't have to be that one.
39:18
It could be anything. But when you make everything
39:21
frictionless, you really remove this core part of being human, I
39:26
think. So that's another opportunity
39:28
where it's it's OK that things are taking maybe a day longer to
39:32
define how we think about responsibility because that will
39:36
maintain what we actually care about for our community, our
39:41
existence, whatever it is. And so that's something that I
39:44
really associate with being human.
39:47
And there are many, but I think that one's a really important
39:49
one. You saying that reminded me of a
39:53
podcast that I was just listening to.
39:55
I can't remember probably 2 weeks ago with this Doctor Who
40:02
has studied over 10,000. Near death experiences And one
40:10
of the things that he said that struck me to the bone was how a
40:16
lot of folks when they have the their near death experiences,
40:20
they have the typical thing that we think of.
40:23
They go into a void and it's just bliss and peace and all the
40:30
good emotions and feelings you could think of.
40:33
And then they come back and it's back to this world, which is a
40:38
bit more dense and there is conflict.
40:42
And one thing that the doctor said or the person, I'll get the
40:48
name real fast and, and tell you exactly what the name was.
40:53
But one thing that he said was there is a uniqueness about us
41:02
being here on this plane and having to deal with conflict
41:07
that you don't get when you are in this heavenly state because
41:11
it is all just bliss and the positive emotions.
41:17
And So what I realized is that us having conflict and the
41:24
adversarial nature that you get in the human experience, that's
41:29
almost like the beauty of being human, being able to go through
41:34
that. And then ideally this is where
41:37
it gets a little bit more flowers and high dye for my
41:40
personal beliefs. That is that you learn to love
41:43
through the conflict and you learn to be able to have those
41:49
positive emotions despite all the adversary.
41:53
Yeah. This concept of like you can't
41:56
can only experience light when you have dark, and you only know
42:01
when things are good because you've also experienced bad.
42:03
If you've only ever had this perfectly curated frictionless
42:08
experience where nothing ever goes wrong and you never have to
42:12
employ critical thinking, then it's not really the most
42:17
exciting and interesting experience.
42:19
And it's also not a robust 1 back to that concept where it's
42:23
but you're not really understanding that there could
42:25
be something that gets in the way of you and what you want.
42:29
And so when you start to face that, you will not have the
42:32
tools to be able to handle them in a human way.
42:35
Like maybe you have an app to handle it, But let's imagine
42:37
that your app breaks and you as a human really handle life
42:42
anymore. If your entire world has been
42:47
curated and made perfect by these solutions, quote UN quote,
42:52
solutions to everything you might need.
42:54
Yeah. So I think.
42:56
Yeah. This idea of having a chance
42:59
right now for as many people as possible, again, back to
43:04
perspective density, to contribute to the conversation
43:07
of what our future will look like, to shape technology in a
43:10
way that protects what we care about, the kinds of experiences
43:14
that we actually enjoy to have that I think that's it's really
43:17
important right now. So the guy's name that I was
43:22
talking about is Doctor Bruce Grayson, just as an FYI, and I
43:28
heard him on the Tim Ferriss podcast.
43:31
I just looked it up and it's studying over 1000 near death
43:36
experiences. I think I said 10,000, actually
43:39
just 1000. That's a.
43:40
Lot still. Still a quite a big number of
43:44
people who have died and come back and it it is great to think
43:49
about this idea of not dying, but the idea of just having as
43:56
many people in the room giving their opinions on the outcome
44:03
that you're trying to shape and having that perspective density
44:07
is very cool to think about. Yeah, this idea also of of being
44:13
a leader in this time is really related to that for myself.
44:17
Like curiosity is such a big part of the kind of leader that
44:21
I am. And because I work with a lot of
44:24
engineers and a lot of technical teams, I'm in this position
44:27
where I can be super curious and ask a lot of questions and I end
44:31
up at a results and in an understanding that is so much
44:35
more rich and dense. Because I've spoken to so many
44:39
people, I really sat through and listened to like, how does this
44:42
work? How are you working about it?
44:44
Like how would you think through this problem?
44:47
And it's not just me having this idea of this is the best
44:50
solution. I want to do this.
44:52
Like just do it. Like I'm not like that at all
44:54
because I don't know a lot of things.
44:57
There are a lot of things I do know, but there are a lot of
44:59
things I don't know. And I saw I really hang on to
45:01
that where I'm like, how can I go into this room assuming I
45:06
know nothing and come out of this with a much stronger
45:09
understanding because I've really taken the time to to talk
45:12
to all these different people. So I think right now what we
45:16
really need is a sense of leadership that is built on
45:20
curiosity and from trying to interact with a lot of different
45:24
folks. One of our other values other
45:26
than perspective density is intellectual generosity.
45:29
I think that's another huge element of this new era for
45:33
sure. If you know something, being
45:35
able to share it with others. If you are in a space and you
45:38
have the opportunity to bring other intelligent people into
45:41
the room and have them share what they're working on, that's
45:44
massive. It's also why I really love the
45:47
MLS community meetups. Like I'm just getting to bring
45:50
people together and be like, hey, how are you thinking about
45:52
this problem? What are you working on?
45:54
Here's the stage, talk through it, and now we can all learn
45:56
from you. I think that's such a
45:59
transformational, important part of this time that we're in.
46:02
Yes, it is such a good excuse to be able to get to talk with and
46:08
hang out with incredible people. It's you learn the trick of why
46:13
I do this podcast. I get to talk with all the.
46:16
People like. Yourself.
46:17
Yeah, totally. I will also give a shout out to
46:21
your newsletter. I think that is awesome and more
46:25
people should be subscribed to it.
46:27
So if anyone wants to subscribe, we will leave a link to that in
46:34
the description. And I think that you're doing
46:37
great work. Thanks for coming on here,
46:39
Allegra. Yeah.
46:40
Thank you so much for that shout out.
46:42
We just hit one month of the newsletter and I'm so proud.
46:45
I'm so happy about it. It was really the most exciting
46:49
year of, of research and and intellectual generosity.
46:53
So thank you so much for the opportunity.