Latest / The Road to Accountable AI / Ravit Dotan: Rethinking AI Ethics

Transcript

Ravit Dotan: Rethinking AI Ethics · The Road to Accountable AI

• 0:09
  Hi. I'm Kevin Werbach. professor of Legal Studies and Business Ethics at the Wharton
• 0:13
  School of the University of Pennsylvania.
• 0:15
  For decades. I've studied emerging technologies from broadband to blockchain.
• 0:19
  Today. AI is promising to transform our world.
• 0:24
  But AI needs accountability mechanisms to ensure it's developed and deployed in
• 0:29
  responsible. safe. and trustworthy ways.
• 0:32
  On this podcast. I speak with the experts leading the charge for accountable ai.
• 0:40
  On this episode. I speak with Ravit Doan.
• 0:43
  She's the founder and CEO of Tech. better. an AI consulting firm and director of
• 0:47
  the Collaborative AI Responsibility Lab at the University of Pittsburgh.
• 0:51
  She has been named one of the 100 billion women in AI ethics and
• 0:54
  won a distinguished paper award at fact. the top AI ethics conference.
• 0:58
  We discuss how her approach to AI ethics and governance
• 1:01
  has evolved across her career.
• 1:03
  The biggest problems holding back the field and the way she's recently
• 1:07
  rethought her entire approach.
• 1:11
  Ravi. welcome.
• 1:12
  Thank you for joining me on the Road to Accountable ai.
• 1:15
  Hi.
• 1:16
  Great to. great to be here.
• 1:18
  Tell me first about your journey into work on AI ethics and governance.
• 1:25
  Yeah. uh. yeah. it's been a long road.
• 1:28
  Um.
• 1:31
  With various milestones. but. uh. I guess officially I switched to. to
• 1:35
  this space in 2021 after I finished my. my PhD. So it just kind of started
• 1:41
  organically consulting to various organizations. mainly tech companies.
• 1:46
  but also investors. local governments. on how to do everything under the umbrella.
• 1:52
  Like develop it responsibly. procure it responsibly. invest in it responsibly.
• 1:58
  Um.
• 1:59
  So that's what I've been doing in the industry.
• 2:01
  But I don't know if I would say that that is exactly my starting point.
• 2:05
  because in 2017 is where I started doing AI ethics within academia.
• 2:10
  So I was doing a PhD in. in philosophy to see Berkeley.
• 2:13
  Um. and that's. uh. when I. I guess. officially started with AI ethics.
• 2:18
  But. um. I'm not even sure that that would be my official starting point because.
• 2:24
  um. how did I fund all this education?
• 2:27
  I worked in tech.
• 2:29
  Um. and so I think it was. what year was that?
• 2:32
  Was when I started my master's. so I don't know. like 2011 something.
• 2:39
  I worked in the first company that was an AI company. so yeah.
• 2:43
  What was it like making the shift from doing a PhD in philosophy
• 2:47
  to working advising companies and other organizations?
• 2:50
  Yeah. night and day. honestly. I maybe I'll. I'll. I'll share an anecdote too.
• 2:56
  To explain how it was like. uh. my field wasn't ethics. uh. but
• 3:01
  of course I knew the fundamentals.
• 3:03
  I taught ethics.
• 3:05
  And one of my friends from the. from my PhD program asked me like a couple
• 3:09
  years in or something he asked me.
• 3:12
  So. um. you do AI ethics. you find yourself using some of the things
• 3:17
  that you've learned. and I said. actually. I never thought of it.
• 3:20
  But. um. it never comes up. not even once.
• 3:24
  Yeah. so I was in a way surprised. but in other ways it makes sense.
• 3:29
  Other than that. it's much more applied. uh. as opposed to theoretical.
• 3:34
  What. what's different in terms of talking to. to corporate leaders about
• 3:38
  these issues versus thinking about them in the more philosophical context?
• 3:43
  Okay. I'll. I'll continue with the same anecdote.
• 3:45
  Yeah.
• 3:45
  Uh. because when I said no. my friend was surprised.
• 3:48
  Yeah. yeah.
• 3:48
  Uh. and he asked me. what do you mean?
• 3:52
  So what happens when you are. you know. on a call mm-hmm.
• 3:56
  With a client?
• 3:57
  Um. and they describe and they maybe they argue about the decision that
• 4:01
  they have to make. that. you know. has like ethical implications.
• 4:05
  Don't you wanna sometimes jump in and say something like. well. what
• 4:08
  you're saying is like virtue theory and according to it. da da. da da.
• 4:12
  But the other person. it's actually consequentialism.
• 4:15
  It's not. they're saying that. da da da da.
• 4:16
  And I laugh.
• 4:17
  No. absolutely not.
• 4:18
  That would be completely out of place.
• 4:20
  Uh. um. because I think the role that philosophy plays is more.
• 4:26
  Laying out options.
• 4:28
  But when we are in a workplace. we wanna make decisions. it's different.
• 4:33
  But there have to be some foundations for making those decisions.
• 4:36
  The decision can't just be what do we think will be in our business interests?
• 4:41
  Uh. at least in the short run. there's gotta be some way of wrestling
• 4:45
  with a difficult issue about some AI implementation that might have
• 4:48
  some risks or some possible harms.
• 4:51
  So how do you help companies in terms of thinking through those things?
• 4:55
  Right.
• 4:56
  Not necessarily by talking about the ethics aspect.
• 4:59
  Yeah.
• 4:59
  Because honestly. often these are clear.
• 5:02
  Mm. they understand we need to do X. but the problem is the roadblocks
• 5:07
  in the way to do X. Um. and I think that they. that is a problem that is
• 5:11
  a bit underappreciated because we do have so many guidelines now. right?
• 5:16
  Um. we do have risks that are more familiar than they were in the past.
• 5:21
  Um. but what I see in companies.
• 5:24
  Is dealing with internal systems. then don't necessarily prioritize that.
• 5:30
  So knowing how to manage the internal dynamics so that the decision that is
• 5:34
  clear to everyone could be executed. and that seems to be more of a challenge.
• 5:39
  Hm.
• 5:41
  And in addition to the. the consulting work. you also have still continued
• 5:44
  to do academic work in this area.
• 5:47
  Yeah.
• 5:47
  How do you balance those different kinds of activities?
• 5:50
  You know. um. it feels very natural.
• 5:54
  Um. and sometimes I'm thinking about it.
• 5:56
  Like when you are in academia. um. traditionally people divide
• 6:00
  it to three types of things.
• 6:01
  Research. teaching. and service.
• 6:04
  Right.
• 6:04
  And it feels kind of the same.
• 6:06
  I do research in some of my time. but instead of teaching. I have consulting.
• 6:11
  Uh. so it feels very familiar in that sense.
• 6:14
  Does the research influence the consulting?
• 6:16
  Uh. or. or conversely the. the consulting influence how you think
• 6:20
  about what to do for research?
• 6:22
  Yeah. definitely.
• 6:22
  Uh. that's one of the big changes that I've made.
• 6:25
  I'm calling it. I don't like the name kind of. um. practical
• 6:30
  philosophy or something like that.
• 6:31
  I prefer the name engaged.
• 6:33
  Hmm.
• 6:33
  Um. I think that research.
• 6:38
  Consulting are best when they inform one another very. very deeply.
• 6:43
  So I am still doing research. but not philosophy of switch to social science.
• 6:50
  Uh. I mean. not only. uh. but that's. I. I collaborate with people.
• 6:53
  In social science or in computer science.
• 6:56
  And then I write papers that help me understand the trends
• 7:01
  that I'm seeing within industry.
• 7:04
  Mm-hmm.
• 7:04
  And think about them in a more systematic way. right?
• 7:08
  Mm-hmm.
• 7:08
  Um. and then I take what I learn from that academic research. and then I use
• 7:13
  it when I work with. when I work with clients. when I. you know. do thought
• 7:18
  leadership. all of those things.
• 7:21
  What are the trends that you're seeing. uh. or have seen over the last several
• 7:25
  years in this area of AI governance?
• 7:27
  Yeah.
• 7:28
  Okay.
• 7:28
  Uh. many. uh. because different angles. uh. different things.
• 7:34
  But. um. here maybe. uh. I would wanna highlight one of them because it's
• 7:38
  my favorite. uh. one of one group that I collaborate with. um. they
• 7:43
  are organizational psychologists. which I think is such a crucial
• 7:47
  dimension of AI governance that is.
• 7:49
  A bit neglected when we see all the frameworks. they are prescribing systems
• 7:56
  that organizations should have. right?
• 7:58
  They should have a risk management framework. right?
• 8:01
  They should. should have an. I don't know. like an AI ethics champion or whatever.
• 8:05
  They say all those things. but they don't truly talk about how do you deal
• 8:10
  with the realities of the organization.
• 8:12
  Mm-hmm.
• 8:12
  And that's exactly.
• 8:15
  What I'm interested in. because that's what's gonna make a FX stick.
• 8:19
  I. you know. early on. I've done many projects.
• 8:22
  That organization is super excited at the beginning. and I'll write
• 8:27
  whatever report that they want or framework or principles. whatever.
• 8:31
  And I think that they are genuinely excited.
• 8:33
  I don't think they're lying. but then it doesn't happen.
• 8:36
  Why it goes back. this organizational reality. apparently
• 8:40
  it's a well known phenomenon.
• 8:42
  Called decoupling that people have studied in organizational psychology when there is
• 8:48
  separation between what the organization says and what the organization does.
• 8:52
  And so a lot of people are critical about that.
• 8:55
  We read about AI ethics washing. and that's what it feels like.
• 8:58
  And. um. yeah. I. I also have critique about it. like it feels uncomfortable.
• 9:03
  um. especially when you are in one of those companies and you are the
• 9:06
  person who's being used. you know?
• 9:08
  As we know dressing.
• 9:09
  Mm-hmm.
• 9:09
  Um. but the more interesting question is what can we do?
• 9:13
  Yeah.
• 9:13
  And right.
• 9:14
  Are there different types?
• 9:16
  And what can we do for each of those different types?
• 9:19
  And so one of the papers that's I had written recently is exactly about that.
• 9:24
  Can we. can we understand the different kinds of AI ethics washing. especially
• 9:29
  using this literature on decoupling.
• 9:31
  It already exists.
• 9:33
  And then can we understand how we can help organizations in different buckets?
• 9:37
  Right.
• 9:38
  And one of the things that I've noticed is that some of the recommendations
• 9:41
  from the well-known frameworks are going to be counterproductive
• 9:47
  depending on the decoupling profile or the AI FX washing profile that
• 9:51
  we see in in the organization.
• 9:55
  Can you give an example of some of the guidance for a
• 9:57
  particular kind of organization?
• 9:59
  Yeah.
• 9:59
  Okay.
• 9:59
  So one practice that's very widespread today is having a committee or a process
• 10:05
  to approve AI use cases or AI applications before. you know. early on. hopefully.
• 10:12
  Mm-hmm.
• 10:12
  Not after a day. have already been. um. you know. ready for deployment.
• 10:17
  Um. especially this trend. I think.
• 10:19
  Has become more and more established in light of the EOAI act because
• 10:24
  companies wanna make sure that they don't have use cases that fall in
• 10:27
  the unacceptable risk category or even in the high risk category.
• 10:31
  They would rather catch it early and kind of push it down.
• 10:34
  Um. and so they have committees or forms that need to be submitted.
• 10:38
  For engineers or the product manager. whoever. to say. this is what
• 10:43
  we're doing and we want approval.
• 10:45
  Right?
• 10:45
  So that's a very. very common process.
• 10:47
  And also a lot of the frameworks recommend exactly something like
• 10:51
  this and. and it does make sense. but what are we seeing in practice?
• 10:55
  Right?
• 10:56
  We're seeing that often these processes are not.
• 11:02
  Implemented very smoothly.
• 11:04
  So it might be that there's inconsistency over time.
• 11:08
  At one point they will be implemented. but later they're not.
• 11:12
  Um. or there might be inconsistency across the organization.
• 11:16
  Some teams do it and some teams don't.
• 11:19
  Or inconsistent even between employees.
• 11:21
  Um. and what we're seeing also is that a lot of employees. they try to avoid it.
• 11:26
  They're like. I know there's this process. but we try.
• 11:31
  Not to have to do it.
• 11:32
  So we'll say that our model. you know. we'll say this is not a new model.
• 11:35
  We'll try to pass it under like a existing model's name. right?
• 11:38
  They'll try to resist.
• 11:39
  They'll try to avoid. and what's worse. we are seeing really tension
• 11:44
  within companies because those who are required to undergo this process are a
• 11:50
  good resentful even sometimes of those teams who are enforcing the processes.
• 11:55
  Right?
• 11:56
  And that's not productive.
• 11:57
  It's not what we want.
• 12:00
  Is there a way to overcome that?
• 12:03
  Yeah.
• 12:04
  Um. so one of the reasons I collaborated with the organizational psychologists
• 12:09
  is that they have already seen this. um. in spaces that are similar. like the.
• 12:14
  I like safety. like physical safety. like in factories and even in privacy.
• 12:19
  Um. and according to the theory that we are working with. uh.
• 12:22
  theory of organizational climate.
• 12:25
  What needs to shift is the climate in the organization.
• 12:28
  So it's not about adding a training or putting someone in charge. right.
• 12:34
  or doing a survey about their risks.
• 12:38
  It's definitely not about adding more layers to this cumbersome process.
• 12:42
  It's not about adding even more strict like enforcement mechanisms.
• 12:46
  Um. so climate is. um.
• 12:51
  How the employees perceive what the organization expects from them.
• 12:55
  Mm-hmm.
• 12:55
  Um. culture is maybe around values or something like that. but the climate
• 12:59
  is what do the employees think that the organization really. really wants?
• 13:04
  Okay.
• 13:04
  Um. so for example. if an organization has like high privacy climate. it
• 13:10
  means that the employees perceive that the organization really
• 13:13
  prioritizes. um. privacy climate.
• 13:17
  Um.
• 13:18
  What does AI ethic climate means?
• 13:21
  Right?
• 13:22
  It means that the organ. like the employees in the organization
• 13:24
  perceive that it's really important to the organization.
• 13:27
  And so you ask. you know. what we can do.
• 13:29
  And the research in those other adjacent fields shows that what we
• 13:33
  need to do is improve the climate.
• 13:35
  That's what we need to do.
• 13:36
  That's what's gonna shift things.
• 13:37
  And it. it may not necessarily mean. you know. like I said before.
• 13:40
  going harder on those policies.
• 13:43
  Although then that leads to the next question. which is how do
• 13:46
  organizations improve the climate?
• 13:47
  Because presumably it's not just announced. we really care about AI ethics.
• 13:52
  That's right.
• 13:52
  Yeah.
• 13:53
  Great question.
• 13:54
  Uh. because it's. it's exactly where our research is going.
• 13:57
  Yeah.
• 13:57
  Another thing that we're seeing from the adjacent fields is the critical role
• 14:01
  that middle management actually placed.
• 14:04
  Mm-hmm.
• 14:04
  So in other fields. we've seen that when. um. the middle
• 14:08
  management engages with. um.
• 14:11
  The kind of the. the staff that's on the ground in a certain way.
• 14:15
  you know. about this topic. like talks about it with them. right?
• 14:19
  Sets a personal example. um. makes it a part of like the day-to-day operations
• 14:24
  of the work that really. really matters.
• 14:27
  Also matters because we're talking about ethics. right?
• 14:29
  Um. it matters whether the leadership of the organization is
• 14:33
  perceived as ethical because it's not the separate topic. right?
• 14:38
  Um. and I've seen it even. even in the interviews that I
• 14:41
  did as part of this research.
• 14:42
  The first one. uh. that we finished already. you know. in some organizations
• 14:46
  they haven't started implementing AI ethics yet at that point. right?
• 14:50
  But employees are sure that they will.
• 14:53
  Why?
• 14:53
  Because they believe that the leadership is ethical. right?
• 14:56
  Even if it's unrelated things they say. they don't lie.
• 15:00
  So I believe they will do. you know. they'll. they'll deal with AI in an
• 15:03
  ethical way. which seems unrelated. but it is when you understand their theory.
• 15:08
  Right.
• 15:08
  'cause it creates the overall climate.
• 15:10
  Yeah.
• 15:10
  Makes sense.
• 15:11
  One of the reasons I really wanted to talk to you was a post on your
• 15:14
  substack recently. the. the AI treasure chest. talking about a. a
• 15:18
  recent pivot that you've made and how you're approaching AI ethics.
• 15:21
  So can you talk more about that?
• 15:24
  Oh yeah.
• 15:24
  Yeah.
• 15:24
  Thank you so much. uh. for inviting me.
• 15:26
  'cause I. this is the thing that I. Talk about all the time now. you know. uh.
• 15:32
  when you get excited about a new topic.
• 15:33
  That's exactly the stage that I'm in.
• 15:35
  Um. so maybe I'll start with. um. things that I've observed. kind of like to add
• 15:41
  to where this conversation had gone.
• 15:43
  I.
• 15:46
  in my work earlier. right around 20 21. 20 22 even. there was so much
• 15:53
  excitement about AI governance.
• 15:55
  Um.
• 15:56
  It was not that difficult to find companies that really wanna try.
• 16:01
  There was a lot of. you know. many conversations about that.
• 16:04
  a lot of talks. all of this.
• 16:06
  It just felt like this is the time. like a boom of AI governance.
• 16:10
  Um. I feel that there's been a decline. um. in the level of interest. and I've
• 16:16
  identified several reasons for it.
• 16:19
  And all of those reasons.
• 16:21
  Of the reasons from my shift. and I'll explain what they are because then I
• 16:24
  can explain the shift that I've done and. and. and why One trend that I've
• 16:30
  noticed. well. I'm seeing those that resistance in. in organizations like
• 16:33
  I talked about. and I think some of it comes from organizational issues.
• 16:37
  Um. but.
• 16:39
  I also think that some changes that happened in regulations
• 16:43
  have contributed to it.
• 16:44
  Um. so maybe ironically. but. um. I think that the EUAI ACT has pushed
• 16:49
  ethics outta the conversation because people became Yeah. I. uh. I kind
• 16:54
  of expected the opposite. but. uh. but it does make sense because the
• 16:57
  conversations that I'm hearing about now are way more focused on compliance.
• 17:02
  Mm-hmm.
• 17:02
  Like that. right?
• 17:03
  Like. like the structure that I've. that I've described.
• 17:05
  They just want compliance.
• 17:07
  They're more specifically compliance with EUAI act.
• 17:09
  It reminds me a little bit of GDPR in privacy in that sense.
• 17:14
  Like the. the only conversations really I'm hearing about privacy
• 17:18
  are not around the ethics on it.
• 17:19
  They're around compliance.
• 17:21
  Mm-hmm.
• 17:21
  And so it feels like right now. at least there is. or it's felt like this for
• 17:26
  the last year. that there's a shift.
• 17:28
  It's way more difficult to talk about ethics to When was the last time a company
• 17:33
  wanted to talk to me about fairness?
• 17:35
  I don't even remember.
• 17:36
  Sure.
• 17:37
  You're just not as interested.
• 17:38
  Um. so that's one shift that happens. just a drop in the interest level of companies.
• 17:44
  Um. another shift that has happened is a booming generative
• 17:48
  AI that started in 2022.
• 17:50
  Um. and with it came two things.
• 17:53
  One. um.
• 17:56
  A topic like ethics is always perceived as a red tape.
• 18:00
  As a barrier. right?
• 18:01
  It's always gonna be like that.
• 18:02
  And maybe it's good even. but it became even more so
• 18:05
  after this generative AI boom.
• 18:07
  Because now when you talk to people about ai. what they wanna
• 18:10
  talk to you about is adoption.
• 18:11
  Adoption.
• 18:12
  Adoption.
• 18:12
  Mm-hmm.
• 18:12
  Why?
• 18:13
  By the way. because it all fails.
• 18:14
  Like we see it right?
• 18:15
  In surveys. except for service made by companies that sell ai.
• 18:18
  But except for them. in all the surveys we're seeing. like there's
• 18:22
  not. you know. recent MIT surveys said that 95% of all AI pilots.
• 18:27
  Don't continue.
• 18:28
  Yeah.
• 18:28
  I wouldn't rely on that particular number.
• 18:30
  That's good.
• 18:31
  But. but there's other data that. that is useful that that particular one. uh.
• 18:34
  you didn't. I posted on. on LinkedIn.
• 18:35
  It's is questionable. but you're right there. there are data points.
• 18:38
  Yeah.
• 18:38
  I mean. I will say that it's not the most rigorous that I've seen. but I
• 18:41
  trusted more than. like I saw Open AI just recently came up with a white paper.
• 18:45
  I was like. yeah. I'm not. no.
• 18:47
  Or even when it's the big four who publish them because they also have a stake in it.
• 18:51
  Right.
• 18:52
  They're trying to sell consulting businesses.
• 18:53
  Um. so at least like. okay. I don't have those interests.
• 18:57
  And I. I can actually see the methodology. which is more than others.
• 19:01
  But anyway. yeah. we see that they don't succeed.
• 19:03
  So I think companies now even more. they just. when I try to talk to
• 19:08
  them about ai. they just wanna hear about how to adopt. you know?
• 19:12
  So that decreases their interest in talking with me about ethics.
• 19:15
  Um. and then the third thing is I think that the. the AI governance space hasn't
• 19:22
  really shifted into how do we govern.
• 19:26
  AI adoption.
• 19:27
  Right?
• 19:27
  Because I think also they have gone more into. um. how do we deal with compliance?
• 19:33
  Um. and then also. I mean. there's always. when we get frameworks from
• 19:39
  big organizations. they usually apply to those huge companies. right?
• 19:43
  Mm-hmm.
• 19:43
  And this is also something that I've seen the smaller companies are like. ah.
• 19:48
  This is against us.
• 19:49
  Um. and so those three things that have happened. and in addition. because what
• 19:53
  I've seen is that. you know. the. the tension that happens within companies.
• 19:58
  all of those things just made me think there's gotta be a better way to do this.
• 20:03
  Um. I don't wanna be that kind of road perceived as. as the roadblock.
• 20:08
  Mm-hmm.
• 20:08
  It's gonna stop their innovation.
• 20:10
  Right.
• 20:11
  Because it's not effective.
• 20:13
  It's becoming less and less effective.
• 20:15
  So I kept thinking. how do I do it differently?
• 20:17
  What can I do that's gonna be different?
• 20:21
  And I realized while there's a challenge here. the one that I
• 20:25
  described. there's also an opportunity.
• 20:28
  Because one of my insights is that we can't be separating the.
• 20:34
  the. the ethics or governance component from the excitement and
• 20:40
  the building new things components.
• 20:42
  Right?
• 20:42
  That's. that's a part of the problem. right?
• 20:45
  So when I think. for example. about how courses go that teach ai. right?
• 20:49
  Maybe at best they'll have a unit at the end.
• 20:52
  Ethics.
• 20:54
  And the unit will be dropped because the course will run out of time and it sends
• 20:57
  exactly. you know. what kind of a signal. that's what happens when you separate.
• 21:01
  And so I thought kind of an analogy to teaching Right.
• 21:05
  Instead of putting it as a unit at the end embedded organically throughout.
• 21:09
  Mm-hmm.
• 21:10
  Right.
• 21:10
  Um. at each and every place.
• 21:12
  And I realized. but wait. I could be not just a person who says to
• 21:17
  people hears how to be responsible.
• 21:20
  Right.
• 21:21
  But to say to people.
• 21:23
  I'm gonna touch with you on your biggest pain point related to ai.
• 21:27
  which is how do we actually adopt it?
• 21:29
  Hmm.
• 21:30
  And as I teach it to you. ingrained in every part of it is going to
• 21:35
  be how do we do it ethically?
• 21:40
  That makes a lot of sense. but isn't there. and inherent tension
• 21:44
  sometimes between what leads to adopting quickly and what leads to
• 21:49
  adopting in the most responsible way.
• 21:51
  Yeah.
• 21:52
  You know. um. definitely.
• 21:55
  But also because of. you know. people are not new to AI now.
• 22:01
  Many people. and they have tried this quick adoption and
• 22:05
  they see that it doesn't work.
• 22:06
  Yeah.
• 22:07
  They see that adoption is not happening.
• 22:09
  People in the organization are not doing it.
• 22:12
  They see that often.
• 22:14
  The outputs are actually very poor. poor quality.
• 22:17
  Like the emperor is a bit naked or a lot naked.
• 22:21
  Right.
• 22:22
  And so people who understand that are ready to have this
• 22:25
  more thoughtful conversation.
• 22:29
  You. you've talked about. uh. the. what you call the chef
• 22:32
  approach to AI governance.
• 22:34
  Yeah.
• 22:34
  What. what does that mean?
• 22:36
  In the public conversation about ai. there's a common
• 22:39
  way that people talk about it.
• 22:41
  Um. it's as if we're supposed to be using AI only to offload tasks and
• 22:47
  say. ai. give me my draft document. give me my draft email. whatever.
• 22:53
  Um. right.
• 22:54
  Like the whole point is to offload because they're trying to convince us
• 22:58
  that those tools can replace us. so it's gonna replace us if we offload.
• 23:02
  So I call this the takeout approach 'cause it's like getting takeout. right?
• 23:06
  When I get takeout. I don't want to deal with it.
• 23:08
  I just want to put in my order and get my food.
• 23:11
  I may not even know how to cook the food.
• 23:13
  I probably don't because I not a really good cook.
• 23:17
  I just wanna get it and as fast as possible.
• 23:20
  Um. and I might revise some things at the end.
• 23:24
  I could add salt. I could move it to a different plate.
• 23:28
  I probably won't. but I could.
• 23:31
  Um. but this. you see. this is how the whole approach. I just. I'm
• 23:34
  using the AI to just get my final up.
• 23:37
  That's the most common approach that I've seen.
• 23:39
  I. I don't think it's. I mean. it's. it's helpful sometimes not discounting it.
• 23:43
  but if it's the only thing that we do where you're just gonna be very limited.
• 23:47
  So created an alternative. and that's what I'm calling the chef approach.
• 23:52
  The main point is to think about processes.
• 23:56
  Instead of thinking of just outputs.
• 23:59
  Okay.
• 23:59
  So. so I'm creating a process to take myself through to
• 24:03
  get to the outcome I want.
• 24:05
  And maybe the best way to explain it is with an example.
• 24:08
  Um. so I'll do an example.
• 24:10
  One of the most common usages for. for AI is brainstorming.
• 24:14
  Um. I want ideas. ai just like give you my ideas.
• 24:18
  Okay.
• 24:19
  Um. and so for example. um.
• 24:23
  Give me a topic that you wanna brainstorm about how to build an AI governance plan.
• 24:29
  Okay. great.
• 24:30
  How to build an AI governance plan.
• 24:32
  Uh. the traditional quote unquote way to do it is to go to the
• 24:36
  chat bot and say. chat bot.
• 24:38
  Tell me. yeah.
• 24:39
  Tell me the way to build an AI governance plan and it'll spit something at
• 24:43
  you and it will be probably cliche.
• 24:46
  And probably include misinformation and probably not incorporating
• 24:51
  your expertise and probably not customize to what you need.
• 24:56
  Um. that's the takeout approach.
• 24:58
  Just like. give me my punti. I want it now.
• 25:00
  I don't wanna worry about it.
• 25:02
  Another way to do it to say. you know. I'm going to build a process
• 25:07
  for me to brainstorm about this topic so it can go something like this.
• 25:11
  You just read instructions in English or whatever language. um.
• 25:15
  to the chat bot and say. chat bot.
• 25:17
  You're gonna take me through a process.
• 25:18
  Now. I'm going to first. like step number one. I'm going to tell you
• 25:23
  something that I wanna be thinking about.
• 25:25
  Two. you are going to ask me three follow up questions to deepen
• 25:30
  my thinking about this topic.
• 25:32
  Three.
• 25:33
  You are going to create a summary of what I just said.
• 25:37
  And then you put that in just like in. into the chat window.
• 25:41
  Even it. it can be this. that simple and then. okay. what's your topic?
• 25:45
  I wanna think about AI governance and here. and then you say. here
• 25:49
  are my initial thoughts. because you already have thoughts mm-hmm.
• 25:51
  On the topic. right?
• 25:52
  And then you get this drill down and then in the end you get something that
• 25:57
  is actually based on your thoughts.
• 25:59
  It's a deeper. you see it's a deeper process.
• 26:01
  Mm-hmm.
• 26:02
  Uh. that you take yourself through.
• 26:05
  And then how does that impact on the. the ethics and governance questions.
• 26:09
  whether it's bias or privacy or accuracy?
• 26:13
  Yeah.
• 26:13
  Yeah.
• 26:13
  Perfect.
• 26:14
  Because once you do it that way. you also get an opportunity to
• 26:17
  add a lot of guidelines inside.
• 26:20
  Hmm.
• 26:20
  Such as. and I always add this.
• 26:22
  Never be biased.
• 26:24
  Yeah. you. you would be amazed how much a sentence like this
• 26:28
  can actually impact your output.
• 26:29
  I will say that it really depends on the chat bot. and in my
• 26:32
  experience. Claude listens more than chat PT and Gemini the least.
• 26:36
  But that is just a. I don't wanna like bad mouth anyone 'cause it's
• 26:39
  been a while since I've tested.
• 26:40
  But anyway. yeah. you can add your guidelines.
• 26:43
  So for example. I created.
• 26:45
  Created a process like this to create cover images for my. for
• 26:50
  my substack newsletter. like you just said. the AI treasure chest.
• 26:53
  Um. and I actually. I just wanted a prompt for the image because I. I have my. my
• 26:58
  favorite image generator that didn't train on stolen data. so I just used that.
• 27:04
  Uh. but I wanted the prompt and I wanted the prompt to include both kind of a
• 27:09
  sentence description of what the image should show. but I also wanted the style.
• 27:14
  When you ask for style. you get really close to. you know.
• 27:18
  copyright infringement risk. right?
• 27:19
  Which. you know. we've. we've seen many complaints about that.
• 27:23
  So I've added a guideline. never suggests styles that name specific people. right?
• 27:30
  If the user ever asks for something like that. refuse and instead
• 27:34
  just offer a general description.
• 27:38
  Got it.
• 27:39
  I wanna go back to. to one thing that you talked about earlier. um. which is
• 27:43
  the. the shift to generative AI in terms of how that changes the environment.
• 27:48
  Uh. what else should we be thinking differently about in this area now that
• 27:53
  we're in a increasingly gen AI world?
• 27:58
  Okay. I'm gonna touch on AI governance. uh. because that's our topic.
• 28:01
  Yeah.
• 28:02
  Um. when we think of all the guidelines. right. that say to engineers what they
• 28:06
  should be doing. the reality is that one of the shifts that engineerings are.
• 28:11
  that engineering is undergoing now is way more incorporation of generated code.
• 28:16
  Mm-hmm.
• 28:18
  Can we give them instructions on how to do that responsibly?
• 28:21
  Because currently it's not really working.
• 28:23
  Um. a research paper that I've seen said that an estimated 40% of generated code
• 28:32
  has cybersecurity infringement issues.
• 28:36
  Um. different papers showed the different numbers. but that's
• 28:39
  kind of like the average.
• 28:40
  It was like a meta overview. I guess.
• 28:43
  Um.
• 28:45
  That's only about cybersecurity.
• 28:46
  I don't know that anyone has checked for the other risks. like bias. for
• 28:51
  example. or. you know. or anything else.
• 28:54
  Like all like the. the umbrella of them.
• 28:56
  Right.
• 28:57
  And also that engineers. even when they see those mistakes. they often
• 29:01
  do not fix them. uh. because they just kind of take what they get.
• 29:06
  Even when they know it's false. they still don't fix it.
• 29:09
  You see that is a new source of AI risks that I don't think
• 29:14
  anyone is really handling.
• 29:17
  But it gets back to the same question you were talking about earlier.
• 29:20
  If the coders are just focused on addressing some problem. having
• 29:25
  to think about cyber risk or something like that. they're not
• 29:29
  necessarily incentivized to do that.
• 29:32
  Yes. exactly.
• 29:34
  Exactly.
• 29:34
  Which is where you see.
• 29:37
  Which is where the chef approach comes in. because what they also don't know how. how
• 29:41
  what to do is actually adopt AI as users.
• 29:44
  Even when you talk to an engineer who's very proficient. right?
• 29:48
  They write AI code.
• 29:49
  Mm-hmm.
• 29:50
  They have not necessarily thought of creating processes for themselves.
• 29:54
  and I've done this with engineers.
• 29:55
  Right?
• 29:55
  Right.
• 29:56
  And then they. they're not getting the output that they want.
• 29:59
  And then you can explain to them. well. instead of expecting. you know. the
• 30:02
  chat bot to just give you your code.
• 30:04
  I can just like spit it at you.
• 30:06
  Take out weight.
• 30:07
  Why don't I tell you. why don't I teach you how to build a process that's
• 30:12
  gonna get you where you wanna go?
• 30:14
  And also by the way. you know you wanna be responsible.
• 30:17
  Let me explain to you how to incorporate those GREs within it.
• 30:22
  Yeah. it's interesting because in. in traditional coding. at least within big
• 30:26
  companies and tech companies. there are whole code quality processes and. and
• 30:30
  mechanisms for doing those cyber reviews.
• 30:32
  You know. you're working at Microsoft.
• 30:34
  They're not gonna put anything in. into production without
• 30:37
  working through those elements.
• 30:39
  But it seems like now when we're in this world. when people can just
• 30:42
  vibe code and. and you know. speak into the. the AI agent. some of that
• 30:47
  potentially gets left by the wayside.
• 30:49
  It sounds like it's part of what you're saying.
• 30:50
  And. and you know. with AI ethics topics. it's even worse because. and I've seen
• 30:54
  this in. in my research in some of the topics. the ones who are responsible for.
• 31:00
  Kind of ringing the bell are the same engineers who are gonna be
• 31:03
  expected to then fix it. and so they just won't bring that down.
• 31:07
  Right.
• 31:08
  All comes back to incentives.
• 31:10
  Yeah. and. and also in resisting that separation between the topics.
• 31:16
  Because one reason that they won't ring it is because they
• 31:18
  don't know how to solve it.
• 31:20
  Right.
• 31:20
  Because those teams from AI governance are often.
• 31:24
  They don't have the bandwidth right.
• 31:26
  To really help with it.
• 31:27
  But what if you can tell to the engineer. listen. I know you wanna be responsible.
• 31:32
  We all do.
• 31:32
  Look. this is who we are.
• 31:34
  Most people want to do it. but they don't because of various constraints.
• 31:38
  Mm-hmm.
• 31:38
  But you can say to them. listen. but let me teach you how.
• 31:42
  If you even just add those guardrails. it's already gonna be better.
• 31:46
  Right.
• 31:46
  Um. and if you tell them. if you teach them how to loop themselves into the
• 31:50
  process so that they can review it better. because when we do this takeout
• 31:55
  mode with ai. we're also not very inclined to actually review it because
• 32:00
  the whole point was to offload it.
• 32:01
  Now you have to like really review this like super long code.
• 32:04
  No. you don't wanna do it.
• 32:07
  What's the most important thing for organizations to do to get started on
• 32:11
  this path of taking responsibility. integrating those functions?
• 32:15
  You know. I think it. I think it starts with teaching with wherever
• 32:21
  you build your employees skill.
• 32:25
  Embedded there. not as a unit at the end that's is gonna be dropped.
• 32:29
  Right.
• 32:30
  But as an organic part of teaching new skills. I think
• 32:33
  that's a great way to begin.
• 32:34
  And also what I'm learning from the organizational psychologists.
• 32:38
  it's all about incorporating middle management. right?
• 32:42
  Well. we need to wrap up. but Ravi. thank you so much.
• 32:44
  It's been a really fascinating conversation.
• 32:46
  Yeah. thanks for having me.
• 32:49
  This has been the Road to Accountable ai.
• 32:51
  If you like what you're hearing. please give us a good review and
• 32:54
  check out my substack for more insights on AI accountability.
• 32:58
  Thank you for listening.
• 33:02
  If you want to go deeper on AI governance. trust and responsibility with me.
• 33:06
  Another distinguished faculty of the world's top business school.
• 33:10
  Sign up for the next cohort of Wharton's Strategies for Accountable
• 33:13
  AI Online Executive education program. featuring live interaction
• 33:17
  with faculty expert interviews and custom designed asynchronous content.
• 33:22
  Join fellow business leaders to learn valuable skills you can
• 33:25
  put to work in your organization.
• 33:27
  Visit exec ed.warden.up.edu/acai for full details.
• 33:35
  I hope to see you there.
• 34:04
  Hi. I'm Kevin Warbeck. professor of Legal Studies and Business Ethics at the Wharton
• 34:07
  School of the University of Pennsylvania.
• 34:10
  For decades. I've studied emerging technologies from broadband to blockchain.
• 34:14
  Today. AI is promising to transform our world.
• 34:19
  But AI needs accountability mechanisms to ensure it's developed and deployed in
• 34:23
  responsible. safe. and trustworthy ways.
• 34:26
  On this podcast. I speak with the experts leading the charge for accountable ai.
• 34:35
  On this episode. I speak with Ravit Doan.
• 34:37
  She's the founder and CEO of Tech. better. an AI consulting firm and director of
• 34:41
  the Collaborative AI Responsibility Lab at the University of Pittsburgh.
• 34:46
  She has been named one of the 100 billion women in AI ethics and
• 34:49
  won a distinguished paper award at fact. the top AI ethics conference.
• 34:53
  We discuss how her approach to AI ethics and governance
• 34:56
  has evolved across her career.
• 34:58
  The biggest problems holding back the field and the way she's recently
• 35:02
  rethought her entire approach.
• 35:06
  Ravi. welcome.
• 35:07
  Thank you for joining me on the Road to Accountable ai.
• 35:10
  Hi.
• 35:11
  Great to. great to be here.
• 35:13
  Tell me first about your journey into work on AI ethics and governance.
• 35:20
  Yeah. uh. yeah. it's been a long road.
• 35:23
  Um.
• 35:26
  With various milestones. but. uh. I guess officially I switched to. to
• 35:30
  this space in 2021 after I finished my. my PhD. So it just kind of started
• 35:36
  organically consulting to various organizations. mainly tech companies.
• 35:41
  but also investors. local governments. on how to do everything under the umbrella.
• 35:47
  Like develop it responsibly. procure it responsibly. invest in it responsibly.
• 35:53
  Um.
• 35:54
  So that's what I've been doing in the industry.
• 35:56
  But I don't know if I would say that that is exactly my starting point.
• 36:00
  because in 2017 is where I started doing AI ethics within academia.
• 36:04
  So I was doing a PhD in. in philosophy to see Berkeley.
• 36:08
  Um. and that's. uh. when I. I guess. officially started with AI ethics.
• 36:13
  But. um. I'm not even sure that that would be my official starting point because.
• 36:19
  um. how did I fund all this education?
• 36:22
  I worked in tech.
• 36:24
  Um. and so I think it was. what year was that?
• 36:27
  Was when I started my master's. so I don't know. like 2011 something.
• 36:34
  I worked in the first company that was an AI company. so yeah.
• 36:38
  What was it like making the shift from doing a PhD in philosophy
• 36:42
  to working advising companies and other organizations?
• 36:45
  Yeah. night and day. honestly. I maybe I'll. I'll. I'll share an anecdote too.
• 36:51
  To explain how it was like. uh. my field wasn't ethics. uh. but
• 36:56
  of course I knew the fundamentals.
• 36:58
  I taught ethics.
• 36:59
  And one of my friends from the. from my PhD program asked me like a couple
• 37:04
  years in or something he asked me.
• 37:07
  So. um. you do AI ethics. you find yourself using some of the things
• 37:12
  that you've learned. and I said. actually. I never thought of it.
• 37:15
  But. um. it never comes up. not even once.
• 37:18
  Yeah. so I was in a way surprised. but in other ways it makes sense.
• 37:24
  Other than that. it's much more applied. uh. as opposed to theoretical.
• 37:29
  What. what's different in terms of talking to. to corporate leaders about
• 37:33
  these issues versus thinking about them in the more philosophical context?
• 37:38
  Okay. I'll. I'll continue with the same anecdote.
• 37:40
  Yeah.
• 37:40
  Uh. because when I said no. my friend was surprised.
• 37:43
  Yeah. yeah.
• 37:43
  Uh. and he asked me. what do you mean?
• 37:47
  So what happens when you are. you know. on a call mm-hmm.
• 37:51
  With a client?
• 37:52
  Um. and they describe and they maybe they argue about the decision that
• 37:56
  they have to make. that. you know. has like ethical implications.
• 38:00
  Don't you wanna sometimes jump in and say something like. well. what
• 38:03
  you're saying is like virtue theory and according to it. da da. da da.
• 38:07
  But the other person. it's actually consequentialism.
• 38:09
  It's not. they're saying that. da da da da.
• 38:11
  And I laugh.
• 38:12
  No. absolutely not.
• 38:13
  That would be completely out of place.
• 38:15
  Uh. um. because I think the role that philosophy plays is more.
• 38:21
  Laying out options.
• 38:22
  But when we are in a workplace. we wanna make decisions. it's different.
• 38:28
  But there have to be some foundations for making those decisions.
• 38:31
  The decision can't just be what do we think will be in our business interests?
• 38:35
  Uh. at least in the short run. there's gotta be some way of wrestling
• 38:40
  with a difficult issue about some AI implementation that might have
• 38:43
  some risks or some possible harms.
• 38:46
  So how do you help companies in terms of thinking through those things?
• 38:50
  Right.
• 38:51
  Not necessarily by talking about the ethics aspect.
• 38:54
  Yeah.
• 38:54
  Because honestly. often these are clear.
• 38:57
  Mm. they understand we need to do X. but the problem is the roadblocks
• 39:02
  in the way to do X. Um. and I think that they. that is a problem that is
• 39:06
  a bit underappreciated because we do have so many guidelines now. right?
• 39:11
  Um. we do have risks that are more familiar than they were in the past.
• 39:16
  Um. but what I see in companies.
• 39:19
  Is dealing with internal systems. then don't necessarily prioritize that.
• 39:25
  So knowing how to manage the internal dynamics so that the decision that is
• 39:29
  clear to everyone could be executed. and that seems to be more of a challenge.
• 39:34
  Hm.
• 39:35
  And in addition to the. the consulting work. you also have still continued
• 39:39
  to do academic work in this area.
• 39:42
  Yeah.
• 39:42
  How do you balance those different kinds of activities?
• 39:45
  You know. um. it feels very natural.
• 39:48
  Um. and sometimes I'm thinking about it.
• 39:51
  Like when you are in academia. um. traditionally people divide
• 39:55
  it to three types of things.
• 39:56
  Research. teaching. and service.
• 39:58
  Right.
• 39:59
  And it feels kind of the same.
• 40:01
  I do research in some of my time. but instead of teaching. I have consulting.
• 40:06
  Uh. so it feels very familiar in that sense.
• 40:09
  Does the research influence the consulting?
• 40:11
  Uh. or. or conversely the. the consulting influence how you think
• 40:15
  about what to do for research?
• 40:17
  Yeah. definitely.
• 40:17
  Uh. that's one of the big changes that I've made.
• 40:20
  I'm calling it. I don't like the name kind of. um. practical
• 40:25
  philosophy or something like that.
• 40:26
  I prefer the name engaged.
• 40:28
  Hmm.
• 40:28
  Um. I think that research.
• 40:32
  Consulting are best when they inform one another very. very deeply.
• 40:38
  So I am still doing research. but not philosophy of switch to social science.
• 40:45
  Uh. I mean. not only. uh. but that's. I. I collaborate with people.
• 40:48
  In social science or in computer science.
• 40:51
  And then I write papers that help me understand the trends
• 40:56
  that I'm seeing within industry.
• 40:58
  Mm-hmm.
• 40:59
  And think about them in a more systematic way. right?
• 41:03
  Mm-hmm.
• 41:03
  Um. and then I take what I learn from that academic research. and then I use
• 41:08
  it when I work with. when I work with clients. when I. you know. do thought
• 41:13
  leadership. all of those things.
• 41:16
  What are the trends that you're seeing. uh. or have seen over the last several
• 41:20
  years in this area of AI governance?
• 41:22
  Yeah.
• 41:23
  Okay.
• 41:23
  Uh. many. uh. because different angles. uh. different things.
• 41:29
  But. um. here maybe. uh. I would wanna highlight one of them because it's
• 41:33
  my favorite. uh. one of one group that I collaborate with. um. they
• 41:37
  are organizational psychologists. which I think is such a crucial
• 41:42
  dimension of AI governance that is.
• 41:44
  A bit neglected when we see all the frameworks. they are prescribing systems
• 41:51
  that organizations should have. right?
• 41:53
  They should have a risk management framework. right?
• 41:56
  They should. should have an. I don't know. like an AI ethics champion or whatever.
• 42:00
  They say all those things. but they don't truly talk about how do you deal
• 42:05
  with the realities of the organization.
• 42:07
  Mm-hmm.
• 42:07
  And that's exactly.
• 42:10
  What I'm interested in. because that's what's gonna make a FX stick.
• 42:14
  I. you know. early on. I've done many projects.
• 42:17
  That organization is super excited at the beginning. and I'll write
• 42:22
  whatever report that they want or framework or principles. whatever.
• 42:26
  And I think that they are genuinely excited.
• 42:28
  I don't think they're lying. but then it doesn't happen.
• 42:31
  Why it goes back. this organizational reality. apparently
• 42:35
  it's a well known phenomenon.
• 42:37
  Called decoupling that people have studied in organizational psychology when there is
• 42:43
  separation between what the organization says and what the organization does.
• 42:47
  And so a lot of people are critical about that.
• 42:50
  We read about AI ethics washing. and that's what it feels like.
• 42:53
  And. um. yeah. I. I also have critique about it. like it feels uncomfortable.
• 42:57
  um. especially when you are in one of those companies and you are the
• 43:00
  person who's being used. you know?
• 43:03
  As we know dressing.
• 43:04
  Mm-hmm.
• 43:04
  Um. but the more interesting question is what can we do?
• 43:08
  Yeah.
• 43:08
  And right.
• 43:09
  Are there different types?
• 43:11
  And what can we do for each of those different types?
• 43:14
  And so one of the papers that's I had written recently is exactly about that.
• 43:19
  Can we. can we understand the different kinds of AI ethics washing. especially
• 43:24
  using this literature on decoupling.
• 43:26
  It already exists.
• 43:28
  And then can we understand how we can help organizations in different buckets?
• 43:32
  Right.
• 43:33
  And one of the things that I've noticed is that some of the recommendations
• 43:36
  from the well-known frameworks are going to be counterproductive
• 43:42
  depending on the decoupling profile or the AI FX washing profile that
• 43:46
  we see in in the organization.
• 43:50
  Can you give an example of some of the guidance for a
• 43:52
  particular kind of organization?
• 43:54
  Yeah.
• 43:54
  Okay.
• 43:54
  So one practice that's very widespread today is having a committee or a process
• 44:00
  to approve AI use cases or AI applications before. you know. early on. hopefully.
• 44:06
  Mm-hmm.
• 44:07
  Not after a day. have already been. um. you know. ready for deployment.
• 44:12
  Um. especially this trend. I think.
• 44:14
  Has become more and more established in light of the EOAI act because
• 44:19
  companies wanna make sure that they don't have use cases that fall in
• 44:22
  the unacceptable risk category or even in the high risk category.
• 44:25
  They would rather catch it early and kind of push it down.
• 44:29
  Um. and so they have committees or forms that need to be submitted.
• 44:33
  For engineers or the product manager. whoever. to say. this is what
• 44:38
  we're doing and we want approval.
• 44:40
  Right?
• 44:40
  So that's a very. very common process.
• 44:42
  And also a lot of the frameworks recommend exactly something like
• 44:45
  this and. and it does make sense. but what are we seeing in practice?
• 44:50
  Right?
• 44:51
  We're seeing that often these processes are not.
• 44:57
  Implemented very smoothly.
• 44:59
  So it might be that there's inconsistency over time.
• 45:03
  At one point they will be implemented. but later they're not.
• 45:07
  Um. or there might be inconsistency across the organization.
• 45:10
  Some teams do it and some teams don't.
• 45:13
  Or inconsistent even between employees.
• 45:16
  Um. and what we're seeing also is that a lot of employees. they try to avoid it.
• 45:21
  They're like. I know there's this process. but we try.
• 45:26
  Not to have to do it.
• 45:27
  So we'll say that our model. you know. we'll say this is not a new model.
• 45:30
  We'll try to pass it under like a existing model's name. right?
• 45:33
  They'll try to resist.
• 45:34
  They'll try to avoid. and what's worse. we are seeing really tension
• 45:39
  within companies because those who are required to undergo this process are a
• 45:44
  good resentful even sometimes of those teams who are enforcing the processes.
• 45:50
  Right?
• 45:50
  And that's not productive.
• 45:52
  It's not what we want.
• 45:55
  Is there a way to overcome that?
• 45:58
  Yeah.
• 45:59
  Um. so one of the reasons I collaborated with the organizational psychologists
• 46:04
  is that they have already seen this. um. in spaces that are similar. like the.
• 46:09
  I like safety. like physical safety. like in factories and even in privacy.
• 46:14
  Um. and according to the theory that we are working with. uh.
• 46:17
  theory of organizational climate.
• 46:20
  What needs to shift is the climate in the organization.
• 46:23
  So it's not about adding a training or putting someone in charge. right.
• 46:29
  or doing a survey about their risks.
• 46:32
  It's definitely not about adding more layers to this cumbersome process.
• 46:37
  It's not about adding even more strict like enforcement mechanisms.
• 46:41
  Um. so climate is. um.
• 46:46
  How the employees perceive what the organization expects from them.
• 46:50
  Mm-hmm.
• 46:50
  Um. culture is maybe around values or something like that. but the climate
• 46:54
  is what do the employees think that the organization really. really wants?
• 46:59
  Okay.
• 46:59
  Um. so for example. if an organization has like high privacy climate. it
• 47:05
  means that the employees perceive that the organization really
• 47:08
  prioritizes. um. privacy climate.
• 47:12
  Um.
• 47:13
  What does AI ethic climate means?
• 47:16
  Right?
• 47:17
  It means that the organ. like the employees in the organization
• 47:19
  perceive that it's really important to the organization.
• 47:22
  And so you ask. you know. what we can do.
• 47:24
  And the research in those other adjacent fields shows that what we
• 47:28
  need to do is improve the climate.
• 47:30
  That's what we need to do.
• 47:31
  That's what's gonna shift things.
• 47:32
  And it. it may not necessarily mean. you know. like I said before.
• 47:35
  going harder on those policies.
• 47:38
  Although then that leads to the next question. which is how do
• 47:41
  organizations improve the climate?
• 47:42
  Because presumably it's not just announced. we really care about AI ethics.
• 47:47
  That's right.
• 47:47
  Yeah.
• 47:48
  Great question.
• 47:49
  Uh. because it's. it's exactly where our research is going.
• 47:52
  Yeah.
• 47:52
  Another thing that we're seeing from the adjacent fields is the critical role
• 47:56
  that middle management actually placed.
• 47:59
  Mm-hmm.
• 47:59
  So in other fields. we've seen that when. um. the middle
• 48:02
  management engages with. um.
• 48:06
  The kind of the. the staff that's on the ground in a certain way.
• 48:10
  you know. about this topic. like talks about it with them. right?
• 48:14
  Sets a personal example. um. makes it a part of like the day-to-day operations
• 48:19
  of the work that really. really matters.
• 48:21
  Also matters because we're talking about ethics. right?
• 48:24
  Um. it matters whether the leadership of the organization is
• 48:28
  perceived as ethical because it's not the separate topic. right?
• 48:33
  Um. and I've seen it even. even in the interviews that I
• 48:36
  did as part of this research.
• 48:37
  The first one. uh. that we finished already. you know. in some organizations
• 48:41
  they haven't started implementing AI ethics yet at that point. right?
• 48:45
  But employees are sure that they will.
• 48:47
  Why?
• 48:48
  Because they believe that the leadership is ethical. right?
• 48:51
  Even if it's unrelated things they say. they don't lie.
• 48:55
  So I believe they will do. you know. they'll. they'll deal with AI in an
• 48:58
  ethical way. which seems unrelated. but it is when you understand their theory.
• 49:03
  Right.
• 49:03
  'cause it creates the overall climate.
• 49:05
  Yeah.
• 49:05
  Makes sense.
• 49:06
  One of the reasons I really wanted to talk to you was a post on your
• 49:09
  substack recently. the. the AI treasure chest. talking about a. a
• 49:13
  recent pivot that you've made and how you're approaching AI ethics.
• 49:16
  So can you talk more about that?
• 49:18
  Oh yeah.
• 49:19
  Yeah.
• 49:19
  Thank you so much. uh. for inviting me.
• 49:21
  'cause I. this is the thing that I. Talk about all the time now. you know. uh.
• 49:27
  when you get excited about a new topic.
• 49:28
  That's exactly the stage that I'm in.
• 49:30
  Um. so maybe I'll start with. um. things that I've observed. kind of like to add
• 49:35
  to where this conversation had gone.
• 49:38
  I.
• 49:41
  in my work earlier. right around 20 21. 20 22 even. there was so much
• 49:48
  excitement about AI governance.
• 49:50
  Um.
• 49:51
  It was not that difficult to find companies that really wanna try.
• 49:55
  There was a lot of. you know. many conversations about that.
• 49:59
  a lot of talks. all of this.
• 50:01
  It just felt like this is the time. like a boom of AI governance.
• 50:05
  Um. I feel that there's been a decline. um. in the level of interest. and I've
• 50:11
  identified several reasons for it.
• 50:14
  And all of those reasons.
• 50:15
  Of the reasons from my shift. and I'll explain what they are because then I
• 50:19
  can explain the shift that I've done and. and. and why One trend that I've
• 50:24
  noticed. well. I'm seeing those that resistance in. in organizations like
• 50:28
  I talked about. and I think some of it comes from organizational issues.
• 50:32
  Um. but.
• 50:34
  I also think that some changes that happened in regulations
• 50:37
  have contributed to it.
• 50:39
  Um. so maybe ironically. but. um. I think that the EUAI ACT has pushed
• 50:44
  ethics outta the conversation because people became Yeah. I. uh. I kind
• 50:49
  of expected the opposite. but. uh. but it does make sense because the
• 50:52
  conversations that I'm hearing about now are way more focused on compliance.
• 50:56
  Mm-hmm.
• 50:57
  Like that. right?
• 50:58
  Like. like the structure that I've. that I've described.
• 51:00
  They just want compliance.
• 51:01
  They're more specifically compliance with EUAI act.
• 51:04
  It reminds me a little bit of GDPR in privacy in that sense.
• 51:09
  Like the. the only conversations really I'm hearing about privacy
• 51:13
  are not around the ethics on it.
• 51:14
  They're around compliance.
• 51:16
  Mm-hmm.
• 51:16
  And so it feels like right now. at least there is. or it's felt like this for
• 51:21
  the last year. that there's a shift.
• 51:23
  It's way more difficult to talk about ethics to When was the last time a company
• 51:28
  wanted to talk to me about fairness?
• 51:29
  I don't even remember.
• 51:31
  Sure.
• 51:31
  You're just not as interested.
• 51:33
  Um. so that's one shift that happens. just a drop in the interest level of companies.
• 51:39
  Um. another shift that has happened is a booming generative
• 51:43
  AI that started in 2022.
• 51:45
  Um. and with it came two things.
• 51:48
  One. um.
• 51:51
  A topic like ethics is always perceived as a red tape.
• 51:54
  As a barrier. right?
• 51:56
  It's always gonna be like that.
• 51:57
  And maybe it's good even. but it became even more so
• 52:00
  after this generative AI boom.
• 52:02
  Because now when you talk to people about ai. what they wanna
• 52:04
  talk to you about is adoption.
• 52:06
  Adoption.
• 52:07
  Adoption.
• 52:07
  Mm-hmm.
• 52:07
  Why?
• 52:08
  By the way. because it all fails.
• 52:09
  Like we see it right?
• 52:10
  In surveys. except for service made by companies that sell ai.
• 52:13
  But except for them. in all the surveys we're seeing. like there's
• 52:17
  not. you know. recent MIT surveys said that 95% of all AI pilots.
• 52:22
  Don't continue.
• 52:23
  Yeah.
• 52:23
  I wouldn't rely on that particular number.
• 52:25
  That's good.
• 52:26
  But. but there's other data that. that is useful that that particular one. uh.
• 52:29
  you didn't. I posted on. on LinkedIn.
• 52:30
  It's is questionable. but you're right there. there are data points.
• 52:33
  Yeah.
• 52:33
  I mean. I will say that it's not the most rigorous that I've seen. but I
• 52:36
  trusted more than. like I saw Open AI just recently came up with a white paper.
• 52:40
  I was like. yeah. I'm not. no.
• 52:42
  Or even when it's the big four who publish them because they also have a stake in it.
• 52:46
  Right.
• 52:46
  They're trying to sell consulting businesses.
• 52:48
  Um. so at least like. okay. I don't have those interests.
• 52:52
  And I. I can actually see the methodology. which is more than others.
• 52:56
  But anyway. yeah. we see that they don't succeed.
• 52:58
  So I think companies now even more. they just. when I try to talk to
• 53:02
  them about ai. they just wanna hear about how to adopt. you know?
• 53:07
  So that decreases their interest in talking with me about ethics.
• 53:10
  Um. and then the third thing is I think that the. the AI governance space hasn't
• 53:17
  really shifted into how do we govern.
• 53:21
  AI adoption.
• 53:22
  Right?
• 53:22
  Because I think also they have gone more into. um. how do we deal with compliance?
• 53:28
  Um. and then also. I mean. there's always. when we get frameworks from
• 53:34
  big organizations. they usually apply to those huge companies. right?
• 53:38
  Mm-hmm.
• 53:38
  And this is also something that I've seen the smaller companies are like. ah.
• 53:43
  This is against us.
• 53:44
  Um. and so those three things that have happened. and in addition. because what
• 53:48
  I've seen is that. you know. the. the tension that happens within companies.
• 53:53
  all of those things just made me think there's gotta be a better way to do this.
• 53:58
  Um. I don't wanna be that kind of road perceived as. as the roadblock.
• 54:03
  Mm-hmm.
• 54:03
  It's gonna stop their innovation.
• 54:05
  Right.
• 54:06
  Because it's not effective.
• 54:07
  It's becoming less and less effective.
• 54:09
  So I kept thinking. how do I do it differently?
• 54:12
  What can I do that's gonna be different?
• 54:15
  And I realized while there's a challenge here. the one that I
• 54:20
  described. there's also an opportunity.
• 54:23
  Because one of my insights is that we can't be separating the.
• 54:29
  the. the ethics or governance component from the excitement and
• 54:35
  the building new things components.
• 54:37
  Right?
• 54:37
  That's. that's a part of the problem. right?
• 54:40
  So when I think. for example. about how courses go that teach ai. right?
• 54:44
  Maybe at best they'll have a unit at the end.
• 54:47
  Ethics.
• 54:48
  And the unit will be dropped because the course will run out of time and it sends
• 54:52
  exactly. you know. what kind of a signal. that's what happens when you separate.
• 54:56
  And so I thought kind of an analogy to teaching Right.
• 54:59
  Instead of putting it as a unit at the end embedded organically throughout.
• 55:04
  Mm-hmm.
• 55:04
  Right.
• 55:05
  Um. at each and every place.
• 55:07
  And I realized. but wait. I could be not just a person who says to
• 55:11
  people hears how to be responsible.
• 55:15
  Right.
• 55:16
  But to say to people.
• 55:18
  I'm gonna touch with you on your biggest pain point related to ai.
• 55:22
  which is how do we actually adopt it?
• 55:24
  Hmm.
• 55:25
  And as I teach it to you. ingrained in every part of it is going to
• 55:30
  be how do we do it ethically?
• 55:34
  That makes a lot of sense. but isn't there. and inherent tension
• 55:39
  sometimes between what leads to adopting quickly and what leads to
• 55:44
  adopting in the most responsible way.
• 55:46
  Yeah.
• 55:46
  You know. um. definitely.
• 55:50
  But also because of. you know. people are not new to AI now.
• 55:56
  Many people. and they have tried this quick adoption and
• 55:59
  they see that it doesn't work.
• 56:01
  Yeah.
• 56:02
  They see that adoption is not happening.
• 56:04
  People in the organization are not doing it.
• 56:07
  They see that often.
• 56:09
  The outputs are actually very poor. poor quality.
• 56:12
  Like the emperor is a bit naked or a lot naked.
• 56:16
  Right.
• 56:17
  And so people who understand that are ready to have this
• 56:20
  more thoughtful conversation.
• 56:24
  You. you've talked about. uh. the. what you call the chef
• 56:27
  approach to AI governance.
• 56:29
  Yeah.
• 56:29
  What. what does that mean?
• 56:31
  In the public conversation about ai. there's a common
• 56:34
  way that people talk about it.
• 56:36
  Um. it's as if we're supposed to be using AI only to offload tasks and
• 56:42
  say. ai. give me my draft document. give me my draft email. whatever.
• 56:48
  Um. right.
• 56:49
  Like the whole point is to offload because they're trying to convince us
• 56:53
  that those tools can replace us. so it's gonna replace us if we offload.
• 56:57
  So I call this the takeout approach 'cause it's like getting takeout. right?
• 57:00
  When I get takeout. I don't want to deal with it.
• 57:03
  I just want to put in my order and get my food.
• 57:06
  I may not even know how to cook the food.
• 57:08
  I probably don't because I not a really good cook.
• 57:12
  I just wanna get it and as fast as possible.
• 57:15
  Um. and I might revise some things at the end.
• 57:19
  I could add salt. I could move it to a different plate.
• 57:22
  I probably won't. but I could.
• 57:26
  Um. but this. you see. this is how the whole approach. I just. I'm
• 57:29
  using the AI to just get my final up.
• 57:32
  That's the most common approach that I've seen.
• 57:34
  I. I don't think it's. I mean. it's. it's helpful sometimes not discounting it.
• 57:38
  but if it's the only thing that we do where you're just gonna be very limited.
• 57:42
  So created an alternative. and that's what I'm calling the chef approach.
• 57:47
  The main point is to think about processes.
• 57:51
  Instead of thinking of just outputs.
• 57:54
  Okay.
• 57:54
  So. so I'm creating a process to take myself through to
• 57:58
  get to the outcome I want.
• 58:00
  And maybe the best way to explain it is with an example.
• 58:03
  Um. so I'll do an example.
• 58:04
  One of the most common usages for. for AI is brainstorming.
• 58:09
  Um. I want ideas. ai just like give you my ideas.
• 58:13
  Okay.
• 58:14
  Um. and so for example. um.
• 58:18
  Give me a topic that you wanna brainstorm about how to build an AI governance plan.
• 58:24
  Okay. great.
• 58:24
  How to build an AI governance plan.
• 58:27
  Uh. the traditional quote unquote way to do it is to go to the
• 58:31
  chat bot and say. chat bot.
• 58:32
  Tell me. yeah.
• 58:34
  Tell me the way to build an AI governance plan and it'll spit something at
• 58:38
  you and it will be probably cliche.
• 58:41
  And probably include misinformation and probably not incorporating
• 58:46
  your expertise and probably not customize to what you need.
• 58:51
  Um. that's the takeout approach.
• 58:53
  Just like. give me my punti. I want it now.
• 58:55
  I don't wanna worry about it.
• 58:57
  Another way to do it to say. you know. I'm going to build a process
• 59:02
  for me to brainstorm about this topic so it can go something like this.
• 59:06
  You just read instructions in English or whatever language. um.
• 59:10
  to the chat bot and say. chat bot.
• 59:12
  You're gonna take me through a process.
• 59:13
  Now. I'm going to first. like step number one. I'm going to tell you
• 59:17
  something that I wanna be thinking about.
• 59:20
  Two. you are going to ask me three follow up questions to deepen
• 59:25
  my thinking about this topic.
• 59:27
  Three.
• 59:28
  You are going to create a summary of what I just said.
• 59:32
  And then you put that in just like in. into the chat window.
• 59:36
  Even it. it can be this. that simple and then. okay. what's your topic?
• 59:40
  I wanna think about AI governance and here. and then you say. here
• 59:43
  are my initial thoughts. because you already have thoughts mm-hmm.
• 59:46
  On the topic. right?
• 59:47
  And then you get this drill down and then in the end you get something that
• 59:52
  is actually based on your thoughts.
• 59:54
  It's a deeper. you see it's a deeper process.
• 59:56
  Mm-hmm.
• 59:57
  Uh. that you take yourself through.
• 1:00:00
  And then how does that impact on the. the ethics and governance questions.
• 1:00:04
  whether it's bias or privacy or accuracy?
• 1:00:08
  Yeah.
• 1:00:08
  Yeah.
• 1:00:08
  Perfect.
• 1:00:09
  Because once you do it that way. you also get an opportunity to
• 1:00:12
  add a lot of guidelines inside.
• 1:00:14
  Hmm.
• 1:00:15
  Such as. and I always add this.
• 1:00:17
  Never be biased.
• 1:00:19
  Yeah. you. you would be amazed how much a sentence like this
• 1:00:23
  can actually impact your output.
• 1:00:24
  I will say that it really depends on the chat bot. and in my
• 1:00:26
  experience. Claude listens more than chat PT and Gemini the least.
• 1:00:31
  But that is just a. I don't wanna like bad mouth anyone 'cause it's
• 1:00:34
  been a while since I've tested.
• 1:00:35
  But anyway. yeah. you can add your guidelines.
• 1:00:37
  So for example. I created.
• 1:00:40
  Created a process like this to create cover images for my. for
• 1:00:45
  my substack newsletter. like you just said. the AI treasure chest.
• 1:00:48
  Um. and I actually. I just wanted a prompt for the image because I. I have my. my
• 1:00:53
  favorite image generator that didn't train on stolen data. so I just used that.
• 1:00:59
  Uh. but I wanted the prompt and I wanted the prompt to include both kind of a
• 1:01:04
  sentence description of what the image should show. but I also wanted the style.
• 1:01:09
  When you ask for style. you get really close to. you know.
• 1:01:12
  copyright infringement risk. right?
• 1:01:14
  Which. you know. we've. we've seen many complaints about that.
• 1:01:18
  So I've added a guideline. never suggests styles that name specific people. right?
• 1:01:24
  If the user ever asks for something like that. refuse and instead
• 1:01:29
  just offer a general description.
• 1:01:33
  Got it.
• 1:01:34
  I wanna go back to. to one thing that you talked about earlier. um. which is
• 1:01:38
  the. the shift to generative AI in terms of how that changes the environment.
• 1:01:43
  Uh. what else should we be thinking differently about in this area now that
• 1:01:48
  we're in a increasingly gen AI world?
• 1:01:52
  Okay. I'm gonna touch on AI governance. uh. because that's our topic.
• 1:01:56
  Yeah.
• 1:01:56
  Um. when we think of all the guidelines. right. that say to engineers what they
• 1:02:01
  should be doing. the reality is that one of the shifts that engineerings are.
• 1:02:06
  that engineering is undergoing now is way more incorporation of generated code.
• 1:02:11
  Mm-hmm.
• 1:02:13
  Can we give them instructions on how to do that responsibly?
• 1:02:16
  Because currently it's not really working.
• 1:02:18
  Um. a research paper that I've seen said that an estimated 40% of generated code
• 1:02:27
  has cybersecurity infringement issues.
• 1:02:31
  Um. different papers showed the different numbers. but that's
• 1:02:34
  kind of like the average.
• 1:02:35
  It was like a meta overview. I guess.
• 1:02:37
  Um.
• 1:02:39
  That's only about cybersecurity.
• 1:02:41
  I don't know that anyone has checked for the other risks. like bias. for
• 1:02:46
  example. or. you know. or anything else.
• 1:02:49
  Like all like the. the umbrella of them.
• 1:02:51
  Right.
• 1:02:51
  And also that engineers. even when they see those mistakes. they often
• 1:02:56
  do not fix them. uh. because they just kind of take what they get.
• 1:03:00
  Even when they know it's false. they still don't fix it.
• 1:03:04
  You see that is a new source of AI risks that I don't think
• 1:03:09
  anyone is really handling.
• 1:03:12
  But it gets back to the same question you were talking about earlier.
• 1:03:15
  If the coders are just focused on addressing some problem. having
• 1:03:20
  to think about cyber risk or something like that. they're not
• 1:03:24
  necessarily incentivized to do that.
• 1:03:27
  Yes. exactly.
• 1:03:28
  Exactly.
• 1:03:29
  Which is where you see.
• 1:03:31
  Which is where the chef approach comes in. because what they also don't know how. how
• 1:03:36
  what to do is actually adopt AI as users.
• 1:03:39
  Even when you talk to an engineer who's very proficient. right?
• 1:03:43
  They write AI code.
• 1:03:44
  Mm-hmm.
• 1:03:44
  They have not necessarily thought of creating processes for themselves.
• 1:03:49
  and I've done this with engineers.
• 1:03:50
  Right?
• 1:03:50
  Right.
• 1:03:51
  And then they. they're not getting the output that they want.
• 1:03:54
  And then you can explain to them. well. instead of expecting. you know. the
• 1:03:57
  chat bot to just give you your code.
• 1:03:59
  I can just like spit it at you.
• 1:04:01
  Take out weight.
• 1:04:02
  Why don't I tell you. why don't I teach you how to build a process that's
• 1:04:07
  gonna get you where you wanna go?
• 1:04:09
  And also by the way. you know you wanna be responsible.
• 1:04:12
  Let me explain to you how to incorporate those GREs within it.
• 1:04:17
  Yeah. it's interesting because in. in traditional coding. at least within big
• 1:04:21
  companies and tech companies. there are whole code quality processes and. and
• 1:04:25
  mechanisms for doing those cyber reviews.
• 1:04:27
  You know. you're working at Microsoft.
• 1:04:29
  They're not gonna put anything in. into production without
• 1:04:32
  working through those elements.
• 1:04:34
  But it seems like now when we're in this world. when people can just
• 1:04:37
  vibe code and. and you know. speak into the. the AI agent. some of that
• 1:04:42
  potentially gets left by the wayside.
• 1:04:44
  It sounds like it's part of what you're saying.
• 1:04:45
  And. and you know. with AI ethics topics. it's even worse because. and I've seen
• 1:04:49
  this in. in my research in some of the topics. the ones who are responsible for.
• 1:04:55
  Kind of ringing the bell are the same engineers who are gonna be
• 1:04:58
  expected to then fix it. and so they just won't bring that down.
• 1:05:02
  Right.
• 1:05:03
  All comes back to incentives.
• 1:05:05
  Yeah. and. and also in resisting that separation between the topics.
• 1:05:11
  Because one reason that they won't ring it is because they
• 1:05:13
  don't know how to solve it.
• 1:05:14
  Right.
• 1:05:15
  Because those teams from AI governance are often.
• 1:05:19
  They don't have the bandwidth right.
• 1:05:21
  To really help with it.
• 1:05:22
  But what if you can tell to the engineer. listen. I know you wanna be responsible.
• 1:05:26
  We all do.
• 1:05:27
  Look. this is who we are.
• 1:05:28
  Most people want to do it. but they don't because of various constraints.
• 1:05:33
  Mm-hmm.
• 1:05:33
  But you can say to them. listen. but let me teach you how.
• 1:05:37
  If you even just add those guardrails. it's already gonna be better.
• 1:05:40
  Right.
• 1:05:41
  Um. and if you tell them. if you teach them how to loop themselves into the
• 1:05:45
  process so that they can review it better. because when we do this takeout
• 1:05:50
  mode with ai. we're also not very inclined to actually review it because
• 1:05:54
  the whole point was to offload it.
• 1:05:56
  Now you have to like really review this like super long code.
• 1:05:59
  No. you don't wanna do it.
• 1:06:02
  What's the most important thing for organizations to do to get started on
• 1:06:06
  this path of taking responsibility. integrating those functions?
• 1:06:10
  You know. I think it. I think it starts with teaching with wherever
• 1:06:16
  you build your employees skill.
• 1:06:20
  Embedded there. not as a unit at the end that's is gonna be dropped.
• 1:06:24
  Right.
• 1:06:25
  But as an organic part of teaching new skills. I think
• 1:06:27
  that's a great way to begin.
• 1:06:29
  And also what I'm learning from the organizational psychologists.
• 1:06:33
  it's all about incorporating middle management. right?
• 1:06:37
  Well. we need to wrap up. but Ravi. thank you so much.
• 1:06:39
  It's been a really fascinating conversation.
• 1:06:41
  Yeah. thanks for having me.
• 1:06:44
  This has been the Road to Accountable ai.
• 1:06:46
  If you like what you're hearing. please give us a good review and
• 1:06:49
  check out my substack for more insights on AI accountability.
• 1:06:52
  Thank you for listening.
• 1:06:57
  If you want to go deeper on AI governance. trust and responsibility with me.
• 1:07:01
  Another distinguished faculty of the world's top business school.
• 1:07:05
  Sign up for the next cohort of Wharton's Strategies for Accountable
• 1:07:08
  AI Online Executive education program. featuring live interaction
• 1:07:12
  with faculty expert interviews and custom designed asynchronous content.
• 1:07:17
  Join fellow business leaders to learn valuable skills you can
• 1:07:20
  put to work in your organization.
• 1:07:22
  Visit exec ed.warden.up.edu/acai for full details.
• 1:07:30
  I hope to see you there.