Latest / The Road to Accountable AI / Medha Bankhwal and Michael Chui: Implementing AI Trust

Transcript

Medha Bankhwal and Michael Chui: Implementing AI Trust · The Road to Accountable AI

• 0:00
  You
• 0:03
  Welcome, really great to have you guys on the podcast.
• 0:06
  Thanks, Kevin.
• 0:06
  Great to be here.
• 0:08
  Thank you.
• 0:08
  Thank you so much.
• 0:09
  So Meta, tell us first, what are the kinds of projects that companies usually hire McKinsey for in the AI?
• 0:21
  So maybe I'll anchor more on the AI trust topics and then I'll pass it on to Michael to give a kind of big picture also view of other AI topics that we work on.
• 0:32
  Within AI trust, I think we've had across the spectrum different kinds of questions that come up.
• 0:39
  And I think it's also evolved over time from earlier versions of when the executive order came out.
• 0:46
  to kind of now when the questions are much more around tactical solutions, evaluations, and actually looking at third party tooling solutions to implement it in a more
• 0:58
  operational way.
• 0:59
  So the range is from, hey, help us think through our overarching AI strategy, thinking through what use cases are the highest value, how do I measure the business value of these
• 1:10
  use cases, how do I run the pilots, and how do I measure the impact of these pilots?
• 1:16
  and how do I know when to scale these solutions?
• 1:18
  And then on the AI trust side, there are also conversations around, okay, what should be the governance model around actually identifying these risks?
• 1:27
  What does, know, in the presence of evolving AI regulations, what does a high risk use case mean versus low risk in my industry context, in my application context?
• 1:41
  And then also thinking about training, education,
• 1:45
  within the executive leaders and also designing it for specific cohorts, for instance, the legal enterprise risk, InfoSec, product leaders, and kind of designing their education
• 1:58
  curriculum as well.
• 1:59
  So I would say that's kind of like a broad spectrum of questions that we've answered over the past many years on the topic.
• 2:10
  I if I just wanted to add, right, I mean, as Maeda was saying, you know, we've embedded this idea of AI trust, responsibility, accountability, and all the work that we do.
• 2:19
  But, I've been around for a while, right?
• 2:21
  I mean, oftentimes McKinsey is considered a strategy shop, right?
• 2:24
  But actually for decades, it's been a minority of the work that we do.
• 2:27
  it's, I barely recognized the firm that I joined, right?
• 2:30
  I mean, so when I joined, had PhD in computer science.
• 2:32
  I was sort of a weird person at the time.
• 2:34
  We do do, as Maeda says, AI strategy work.
• 2:38
  Right?
• 2:38
  What should I do for, how do I deploy AI within my enterprise?
• 2:42
  And of course, you know, we also serve investors who think of AI as an investment theme.
• 2:46
  That's the kind of work we've done literally for almost a hundred years now.
• 2:51
  But now, like, you know, like I said, it was a bit of a weird person when I arrived.
• 2:56
  We have all these people who are hands on keyboard.
• 2:58
  We're never be a, you know, a system integrator, right?
• 3:00
  But not only do we provide advice, we're oftentimes
• 3:04
  you know, we describe ourselves as an impact partner, a shoulder to shoulder with our clients developing not only the technical applications and embedding AI trust and
• 3:13
  responsibility, but building businesses with them.
• 3:15
  And, know, it might be staffed with some McKinsey people.
• 3:18
  And then over time, you know, it turns out to be a completely, you know, a client driven business.
• 3:23
  so that the range of things we look at is incredibly wide and the ability to embed AI trust at the beginning and throughout the process is something that
• 3:33
  our clients really demand.
• 3:37
  And Michael, as you said, you've been in this field for a while, like me.
• 3:41
  We've seen many waves of technology development and hype cycles.
• 3:46
  What's different about this AI moment today?
• 3:50
  Yeah, we've known each other for a while, but let's not go on about how old we both are.
• 3:55
  Look, it is a remarkable point in time, I think, where we are now.
• 3:59
  You know, there's a somewhat famous paper that was published that said GPTs are GPTs.
• 4:04
  It's a funny pun, right?
• 4:05
  But GPTs and the second part being general purpose technologies.
• 4:09
  And so one of the things that's remarkable, of course, the technology is remarkable.
• 4:13
  All of us had that experience when we first used chat GPTs.
• 4:16
  wait, wow, look at all these things.
• 4:19
  And we continue to have those sorts of experiences.
• 4:21
  But it's beyond those experiences.
• 4:23
  It's not just the breadth, but it is the breadth.
• 4:26
  There's no industry where this stuff doesn't make a difference.
• 4:30
  There's no business function where you can't use AI to increase your performance.
• 4:34
  There's no role where it doesn't.
• 4:36
  But I think one of the interesting things, if I really get down to the experience or maybe even the political economy of it,
• 4:43
  Here at Afford, lot of the technologies that affected people's jobs on a day-to-day basis primarily affected frontline workers, oftentimes people who were paid the least in an
• 4:52
  organization, oftentimes the roles that had the lowest requirements for education.
• 4:59
  labor economists call it low skill.
• 5:01
  I think that's a terrible term, but what they mean is relatively fewer years in formal schooling.
• 5:09
  But when we did some research, we published this paper, The Economic Potential of Generative AI, and we looked at the roles in which the generative AI technologies
• 5:19
  particularly had the most differential impact.
• 5:21
  And it was almost the exact reverse of what the labor economists called skill bias technological change.
• 5:27
  It was the people who have graduate degrees.
• 5:29
  It was the people you paid most in the organization.
• 5:32
  And so suddenly the people in places of power and privilege say, wait, this affects me too?
• 5:37
  And so suddenly.
• 5:38
  that raises the salience of these technologies and it has, right?
• 5:43
  Because now the entire organization, we talk about transformational technological change.
• 5:47
  This means transforming not just the frontline work, but the entire organization.
• 5:52
  And as a result, I think it is really a transformative moment in terms of the potential impact these technologies can have, not just because the technology is advancing, it
• 6:01
  really changes the way we think about an organization if we operate it and the way that, you
• 6:07
  we all will do our work.
• 6:10
  Meta, does that transformative aspect of generative AI impact on how firms think about trust and responsibility?
• 6:19
  Yeah, so I think there I've had multiple conversations with enterprises where they do recognize that to Michael's point, there is tremendous value in integrating AI into their
• 6:33
  solutions, right?
• 6:34
  And they also recognize that trust is a major barrier to adoption.
• 6:41
  So for instance, we recently did a survey with about
• 6:45
  you know, over 760 organizations in about 38 countries on their responsible AI practices.
• 6:53
  And we did hear from them about how there is, you know, kind of clear gaps in their current processes of audits or third party controls, or how trust will be the kind of big
• 7:07
  thing that will prevent them from adopting AI solutions.
• 7:11
  And...
• 7:12
  On the flip side of it, there is also data to show that the AI assurance market or the markets for solutions for these problems is consistently growing.
• 7:23
  So one of the nuances of that fact is that it kind of varies based on industries.
• 7:30
  So when we ask the question to, for instance, the technology companies or pharmaceutical companies in terms of what their spend was going to be,
• 7:40
  you know, in 2025 on operationalizing AI solutions, the range was around 10 to 13 million.
• 7:50
  And the same question if you ask for kind of retail and some of the other industries, it's low around 5 million.
• 7:56
  But the, and that is the average spend.
• 7:58
  So the question is also which industries will focus much more on trust as a factor.
• 8:05
  And the other thing that I wanted to highlight within this space is
• 8:10
  Also, how does it disrupt the startup ecosystem in terms of developing solutions?
• 8:15
  So one of the big things that has come up in conversations is the need for domain-specific, industry-specific, and use case-specific standardized third-party
• 8:26
  evaluations.
• 8:28
  In the absence of regulations, or when organizations are thinking about either self-regulations or third-party reliance, what solutions exist in the space?
• 8:37
  And then the second question is,
• 8:39
  how would evaluations vary as capabilities evolve?
• 8:42
  So evaluations for text space or chat models, there are some that exist in terms of benchmarks, but what would that mean when we start talking about multimodal or multi-agent
• 8:52
  systems?
• 8:53
  And then the third piece is also thinking about evaluations in the enterprise context at a broad spectrum.
• 8:59
  So thinking not just about safety, but thinking about safety, accuracy, correctness, or different dimensions together.
• 9:07
  And maybe at some...
• 9:08
  point, even in competition with each other.
• 9:11
  And so how would enterprises make selection when you're picking models across these multiple dimensions and what is going to be the trade off for you?
• 9:19
  So I think those are some of the big kind of open questions that enterprises think about when trying to think about the value that Michael highlighted and grounding it on the
• 9:30
  reality of trust.
• 9:33
  You mentioned that the survey showed there was a divergence in terms of what firms are investing in AI deployment.
• 9:40
  Does the investment or the focus on governance or trust track that or is there a divergence in terms of which firms are more concerned about the need to deal with some of
• 9:51
  those responsible AI issues?
• 9:56
  So just to understand the question, is it more around what kind of industries are prioritizing investments?
• 10:04
  Yeah, can you characterize what kinds of firms are more focused on the AI trust issues?
• 10:12
  So I think there is a good correlation between enterprises that are AI forward or kind of have been on the frontline and also enterprises that are heavily investing in responsible
• 10:24
  AI, which is probably an obvious conclusion, but we do see that as an outcome from the survey.
• 10:32
  I think the second thing, which is actually very interesting to me is also geographic distinctions in terms of how different parts of the world
• 10:42
  are investing in AI governance.
• 10:44
  And one of the things that we found that India and US are at the forefront of our AI maturity or AI trust maturity, and their scores were about 23 % and 19 % kind of
• 10:56
  respectively higher than the average scores.
• 10:58
  So there is also merit in understanding why these geographies forward.
• 11:03
  And I can wear my, I'm Indian, so can wear like kind of my India hat.
• 11:07
  And one of the things that was top of mind for me when we saw those outcomes is,
• 11:12
  India has a lot of policy think tanks, industry consortiums that have been consistently publishing responsible AI principles, AI procurement guidelines for public sector.
• 11:25
  And the narrative of AI adoption has been linked with the narrative of responsible AI.
• 11:33
  And I think understanding how different governments are taking that approach will become as important.
• 11:40
  as understanding how different industries are taking the approach to AI trust as well.
• 11:45
  just going to stop a sec.
• 11:46
  We're getting a little bit of lag from you.
• 11:50
  I mean, it'll probably be OK when we get the recording, because it's coming down locally.
• 11:53
  But it's cutting out occasionally.
• 11:57
  Do you want me to repeat that answer?
• 11:58
  I'm happy to do it.
• 11:59
  I the answer was totally fine.
• 12:00
  We're just towards the end and I'm pretty sure when we get the local recording will be just fine.
• 12:03
  But yeah, I don't know if there's any, it may just be something with the connection.
• 12:08
  yeah, I'll let you know if it's a problem.
• 12:10
  You also froze for me for a second.
• 12:11
  So I don't know.
• 12:12
  Maybe it's my internet.
• 12:13
  Yeah, okay.
• 12:14
  It's about, mean, yeah.
• 12:15
  on?
• 12:18
  Is your VPN on?
• 12:21
  I think so, yes.
• 12:23
  Could turn it off.
• 12:26
  We'll probably lose the connection, though.
• 12:28
  So I don't know, Kevin, if we should do that.
• 12:30
  OK.
• 12:30
  I'll just let you know if there's anyone that I think is a problem.
• 12:33
  That was just at the end.
• 12:34
  And again, I'm pretty sure when we get the local recordings, it'll be fine and he can clean it up.
• 12:38
  So yeah, I'll let you know.
• 12:39
  I definitely wouldn't need to record it.
• 12:41
  OK, so I'll come back and answer Michael's question.
• 12:47
  No, was plenty.
• 12:50
  That was plenty.
• 12:50
  No, don't worry about it.
• 12:51
  Totally fine.
• 12:52
  OK.
• 12:54
  Michael, what's your sense for the enterprises you're dealing with in terms of the maturity stage they're at with adoption?
• 13:02
  Are they still trying to find those killer use cases or are they now at the next stage beyond identifying?
• 13:11
  Yeah, mean, a few things.
• 13:12
  If we would just look overall at AI and generative AI adoption in addition to AI trust and responsible AI as Mita was talking about.
• 13:20
  Look, for organization of a certain size, almost everyone is doing something.
• 13:26
  I mean, we've been surveying thousands of executives around the world in terms of how much they're using AI and when they're using AI.
• 13:36
  And in the discussions with executives, see that it is...
• 13:41
  it's very difficult to find an organization that isn't doing something.
• 13:46
  But again, one of those lessons from age is oftentimes the investment in technology precedes capturing real value from it.
• 13:56
  How many CFOs are talking about how many cents per share they've added when they're doing a quarterly earnings report?
• 14:02
  Very few at this point as a result of generative AI.
• 14:05
  But many organizations are doing something.
• 14:07
  So I think everyone is at the stage of recognizing there's value here.
• 14:11
  And in fact, real value is being created oftentimes within a business function.
• 14:16
  So whether or not it's customer service, for instance.
• 14:20
  If you look at every company is a software company now.
• 14:24
  we're seeing software languages.
• 14:28
  You talk about large language models.
• 14:29
  Now you can just ask it to write some code.
• 14:32
  And it really does.
• 14:34
  recently heard this term vibe coding, right?
• 14:38
  People who aren't coders at all are just, you know, asking it to write code for you and having it, you know, do different things, right?
• 14:44
  So that technology is really moving forward, right?
• 14:46
  And so, and again, in marketing, being able to create either brand marketing or personalized marketing, these things really are, you know, going into production.
• 14:55
  But with regard to scale, we're very, very early.
• 14:58
  And again, you know, there's the technology.
• 15:00
  There's piloting or experiments, and then there's really capturing value.
• 15:04
  And there's a lot of work that has to be done.
• 15:07
  There are a lot of time and effort that has to go into that.
• 15:10
  so I think most organizations of a certain size, again, we differentially look at the largest organizations or the most complex organizations.
• 15:19
  They're in that process of saying, look, what works and then how do we achieve scale?
• 15:23
  And achieving scale, again, is, yes, there's a lot of fun technology stuff about achieving scale.
• 15:27
  But then what do you have to do organizationally?
• 15:29
  How do you change the incentives?
• 15:31
  How do you train and re-skill a lot of those?
• 15:34
  And how do you change the process?
• 15:36
  Again, in some of our surveying, we found the organizations that are refashioning their processes, the ones that are really rewiring their processes are the ones that capture the
• 15:45
  most value.
• 15:46
  But that, again, in some ways is quite disruptive to your organization to do that.
• 15:50
  And so I think that's where we're oftentimes seeing that.
• 15:53
  And it's interesting, we do see these differences by sector or industry or geography as Minda was talking about, but we also within an industry and within a geography, we often
• 16:03
  find big differences with regard to the leaders versus laggards.
• 16:08
  And so oftentimes you have to look at the individual enterprise level to say, how far are you, even though we could say healthcare is here and industrial is here or what have you.
• 16:16
  And those differences are real.
• 16:19
  But then oftentimes we see lots of individual variation about the sophistication and maturity of an organization in identifying the risks that they need to deal with, the
• 16:28
  biases they might not do, or the degree of trust that their employees, their business partners, and their customers have in these systems that they're deploying.
• 16:39
  Yeah, so Meta, given that uncertainty and the flux that's happening, how do you work with the clients to see the value in making those investments specifically in the responsible
• 16:50
  AI or AI governance activity?
• 16:53
  Yeah, so I think, you know, the good thing about working in AI trust is that there are a large variety of organizations, multi-stakeholder organizations that have come up with
• 17:06
  frameworks that exist as a foundation layer on which you can base how you work with organizations.
• 17:12
  And, you know, some examples of that would be, for instance, NIST RMF or ISO, and these provide like kind of guidebooks as a starting point.
• 17:22
  When we work with organizations, I think we look at it in multiple tiers of how these frameworks need to be adapted.
• 17:31
  And the frameworks need to be adapted at the industry level, at the organization level, and then at the application or the use case level.
• 17:39
  So at the industry level, there are some industries that have high risk exposure, right?
• 17:44
  Like if you think about the security incidents or if you look at the AI incident database or GDPR fines.
• 17:50
  you will see that some of the industries and the financial services are just more highly represented than other industries.
• 17:57
  And so you have to kind of customize your solution to that industry's risk exposure as well.
• 18:03
  I think the second dimension is where is the organization at in terms of its own readiness?
• 18:08
  And we measure it in different ways, right?
• 18:10
  We measure it in the point of view of their overall RAI approach.
• 18:15
  Do they have an understanding of which risks matter to them as an organization?
• 18:20
  Do they have an AI governance body that is actually responsible for adapting it to regulations, adapting it to evolving AI capabilities?
• 18:31
  We look at the second dimension, which is the processes that they have.
• 18:35
  Do people know what their roles are across the entire AI development lifecycle?
• 18:39
  Or is it kind of just one team that is working on innovating and deploying?
• 18:43
  And then I think the third piece is around the...
• 18:48
  the tooling solutions and the technology that exists.
• 18:51
  So you have to think about how are you evaluating your models?
• 18:54
  How frequently you're evaluating your models?
• 18:57
  Which tools do you have in place?
• 18:58
  And are those tools reliable?
• 19:01
  Because a lot of tools right now exist are probably in startup phase.
• 19:05
  And you have to think about something which is robust and reliable in the longer run.
• 19:10
  And then the third piece is change management and training.
• 19:12
  So those are the four ways that you think about an organizational readiness.
• 19:17
  on the dimension of responsible AI.
• 19:19
  And then I think we kind of start looking at the use cases.
• 19:22
  you know, for instance, are you deploying a generative AI tool that is summarizing your meeting notes?
• 19:30
  And what would the assessment of risk look like in that dimension versus if the same tool is helping your customers make investment decisions on an app, on a bank?
• 19:39
  And the kind of risk associated with each is kind of very different levels.
• 19:45
  And the controls you need for those two are also very different.
• 19:47
  So we then kind of start adapting it to the use cases that the organization has identified that are of high value in terms of assessing the risk parameters.
• 19:57
  And for each of those risks, what kind of controls do they already have in place and what do they not have in place that we can supplement with?
• 20:04
  So I think it's a longer drawn process.
• 20:07
  And along that process, it's also about bringing together different stakeholders within organizations.
• 20:13
  you do need to have the legal team and the enterprise risk or the procurement team or the product team and all of them in the same room.
• 20:20
  And a lot of it is being able to translate to what different people are saying.
• 20:26
  But if there's two companies that are in broadly the same industry and doing broadly the same use case, do you wind up with the same structure and process for AI trust or is it
• 20:39
  more granular to the particular organization?
• 20:42
  I think it is very much grounded on their current state and their aspiration for their future state.
• 20:50
  So one example would be that, you know, we have worked with organizations where there isn't an understanding of what does the risk mean at the level of their use case that
• 21:04
  they're deploying.
• 21:05
  And then we have also worked with organizations that have a highly mature framework.
• 21:11
  of architectural board, architectural review board, or having, you know, the kind of the hiring a product general counsel to actually write product policies that are adapted to EU
• 21:23
  AI Act.
• 21:24
  And you kind of see a whole spectrum of maturity of trust as well.
• 21:29
  And so you kind of have to anchor and meet them where they're at.
• 21:32
  And that's how it varies, even if they're in the same industry, and even if they're kind of broadly deploying the same use cases.
• 21:40
  And Michael, you talked about, McKinsey now has this range of capabilities, not just doing strategy, but are there still distinctive ways in the AI area that your firm would engage
• 21:53
  with companies doing these kinds of projects?
• 21:55
  Yeah, I mean, you know, I've been talking about McKinsey's history, right?
• 21:59
  I mean, people again think of strategy firm.
• 22:02
  At one point, we started doing operations work.
• 22:04
  And you know, sometimes we're like, what are you doing on shop floors?
• 22:09
  That's not us, right?
• 22:10
  And then, but that actually became something that we were able to do.
• 22:13
  We can go from the shop floor to the boardroom and make that connection.
• 22:17
  And I think that's one of the things that, again, it's not just that we have these people who have these incredible technical skills.
• 22:23
  but it connects it with a corporate strategy.
• 22:26
  And from an AI trust standpoint, right?
• 22:29
  It's easy to think about this as a risk management question, i.e.
• 22:33
  avoiding the bad stuff, right?
• 22:34
  I mean, there are all kinds of risks you can identify, like we need to guard against it, all those sorts of things.
• 22:40
  But if you frame it as AI trust, then as Maita was talking about, that's part of your value proposition to your employees, but it's part of your value proposition to your
• 22:49
  customers too.
• 22:50
  And so there's an upside to doing this stuff well.
• 22:53
  There is an upside to avoiding bad things happening, there's an upside to avoiding cyber risk, there's an upside to avoiding lawsuits because of bias, all those sorts of things.
• 23:04
  But there's also real upside in terms of if your customers have trust in you, if your business partners have trust in you, if your employees have trust in you, then you can get
• 23:15
  more done.
• 23:16
  Then you can actually create value for them and that value, part of that value will accrue to your company.
• 23:21
  I think that's part of the approach.
• 23:23
  Again, people often talk about from the beginning or AI trust or responsibility embedded in everything.
• 23:31
  Yes, but for what purpose?
• 23:33
  And being able to understand what the goal of the business is and the metrics and tying those both on the upside and as well as avoided downside, I think are some of the things
• 23:43
  that distinguish what we do from someone who's just, number one, just executing on the technical stuff or number two,
• 23:51
  only looking at this as a way to avoid the negative impacts.
• 23:56
  We view this as a way, holistically, as driving the business forward.
• 24:02
  Yeah, this goes back, I guess, to what I was asking you, Metta, before about incentives.
• 24:06
  Certainly everyone, you know, it's appealing to say this investment in AI trust or governance or responsibility will actually give you some upsides, some benefit.
• 24:17
  Is it possible, though, to measure that?
• 24:21
  Can we actually demonstrate that or can you actually demonstrate that to firms?
• 24:26
  Because obviously that's the ideal.
• 24:29
  Michael, you want to go first?
• 24:30
  I'm curious of both of your takes on this.
• 24:31
  Yeah, I mean, again, there's as for many things, the question about attribution, right?
• 24:37
  You look at what your EPS was and what all are the things that went into it.
• 24:42
  certainly on the negative side, it's very obvious, right?
• 24:45
  When negative things happen, there's a cyber breach or whatever, and suddenly your revenue goes down or your market cap goes down because the reputational hits and all those things
• 24:55
  are definitely true.
• 24:56
  But you can also do other things to measure.
• 25:00
  There are these correlations between how much do you trust this brand?
• 25:06
  Did you use the following application?
• 25:09
  As I said, for instance, if you're getting some sort of advice, financial advice, or you're getting a next product to buy recommendation, how helpful was that to you?
• 25:18
  Those sorts of things, we can find correlations with those as well to revenue, for instance.
• 25:24
  And so again, it is hard to do attribution.
• 25:28
  But we do see these types of things really making a difference.
• 25:34
  Yeah, I think the one thing I would add is that organizations do think about quote unquote the ROI of REI, right?
• 25:45
  Like what is my return?
• 25:47
  I you could look at it in two ways.
• 25:50
  You could look at it to Michael's point in terms of what are the kind of GDPR fines or the EU AI Act regulatory fines or the incidents that you're avoiding that is actually helping
• 26:00
  benefit your brand.
• 26:03
  and also helping you from financial loss point of view.
• 26:07
  I think the second way you could look at it in terms of actually evaluating is when you conduct evaluations.
• 26:14
  So if you do actually do red teaming of models or if you actually make selections of models based on benchmarks that exist, whether it's MMLU or it's the safety benchmarks, AI
• 26:27
  illuminate, et cetera.
• 26:28
  How do you, you know, that's a measurable way to look at what is the problem that you're getting into.
• 26:33
  And from a data layer perspective, you know, there are solutions that exist.
• 26:37
  For instance, the data nutrition label that tells you what kind of data is going into the model.
• 26:41
  So I think there are measurable ways to look at the input and there are measurable ways to look at the output from the downside perspective.
• 26:51
  But there is also a measurable way to look at it from a value perspective, but that is self-decided in terms of what is the value that you're actually aspiring to achieve from,
• 26:59
  which is your own organizational risk appetite as well.
• 27:03
  Yeah, so we're talking about, you know, to being able to explain and understand the benefits of these kinds of investments.
• 27:10
  I want to ask you, Metta, about something that you've written about, which is explaining the impact and the causes of AI decisions, the whole area of explainability and
• 27:21
  interpretability.
• 27:24
  Why is that important for trust in AI?
• 27:28
  And where are we in terms of the capability, especially with
• 27:32
  deep learning systems and generative AI to get a good understanding of why certain outputs are resulting.
• 27:42
  Yeah, so I think it's pretty much evolving.
• 27:45
  I would, you even before answering that question, make a distinction between, and we do talk about it in terms of what is explainability, what is interpretability, and what is
• 27:56
  transparency.
• 27:57
  And I think sometimes people conflate all of them.
• 28:00
  So explainability is kind of thinking about it in terms of when a model is sort of a block.
• 28:07
  a black box and you don't understand what kind of input leads to what kind of output.
• 28:11
  And can you explain that in a human understandable way to the end user, right?
• 28:16
  Interpretability is actually understanding the internal workings of the model and embedding that aspect in the development of models.
• 28:24
  So you're not thinking about it once the model is deployed, but it's actually embedded in training of the model.
• 28:30
  And then when you're thinking about it from the prospect of transparency, it's kind of much more broader, which is
• 28:36
  once you're deploying the model, how are you informing the users where the model is used and what are the harms associated with the model?
• 28:43
  And then even broader approach to transparency, which I have been reading more recently in a lot of journals, is transparency in the format of disclosing the training data sets or
• 28:56
  weights or actually the idea of openness in the model ecosystem.
• 29:00
  And I think there is that distinction in the different contexts.
• 29:07
  I think the explainability aspect is very important just in the context of one, winning trust of not only your consumers, but it is also important from a regulatory disclosure
• 29:19
  perspective.
• 29:20
  And there are a lot of tools that exist.
• 29:23
  We talk about it from the point of view of how do you think about explainability in a human-centric way?
• 29:30
  And so how do you kind of think about explainability in terms of the end user?
• 29:34
  If there is an end user,
• 29:36
  that is a government employee or a regulator, you have to be able to understand the model so that it displays that you are complying with the regulation.
• 29:45
  But if you're thinking about explainability from the point of view of your internal business leader, they need to be able to understand how the model is making decisions and
• 29:56
  how it leads to kind of business insights.
• 29:58
  And so I think that aspect of human centricity where explainability is
• 30:03
  designed to the context of the different stakeholders who interact with the model is probably not as evolved as it can be.
• 30:12
  And I think there's a lot of opportunity and scope over there.
• 30:15
  Got it.
• 30:17
  Michael, what's the biggest or what are the biggest stumbling blocks that you run into with clients in these areas?
• 30:25
  Everything.
• 30:25
  mean, to a certain extent, there's this old saying that history doesn't repeat itself, but it rhymes.
• 30:35
  And so I think a lot of the things that we've seen before are true here, too.
• 30:40
  And look, if you just start at a funnel of understanding, just being aware of some of the challenges, you're not going to manage for an AI trust or a responsibility or safety
• 30:53
  question if you don't know that it's a problem yet.
• 30:56
  And so, you know, for a lot of people just understanding, are things, issues, number one, to be addressed.
• 31:03
  then number two, know, Mayda was just talking about all the things you can do with interpretability, transparency, explainability, not knowing that there, what are the full
• 31:11
  range of different ways in which you can address some of those topics, issues, questions.
• 31:16
  And by the way, some of those are technical.
• 31:19
  You know, there are these tools that we can bring to bear.
• 31:22
  But if you talk about hallucination, for instance, as another challenge around a lot of generative AI, again, I'm sure you've talked about this too, right?
• 31:32
  But if you view the performance of these things as being like a summer intern or somewhere before they go to work or whatever, we still have ways to make workers productive even if
• 31:47
  they're a new joiner.
• 31:49
  And what do we do?
• 31:50
  mean, yeah, they're
• 31:51
  technologies you can use, but also we have management and we have all these other human ways in which we can manage the way that different outputs are used, again, for safety,
• 32:02
  for accuracy, for bias, and for testing.
• 32:06
  I mean, it is funny too, because there's a linguistic thing that happens.
• 32:10
  mean, for many years we talked about benchmarks, but now around here where I live in San Francisco, evals.
• 32:15
  And you know that if someone says eval, they actually have a type of sophistication.
• 32:21
  And so there are these linguistic markers of someone understanding something more.
• 32:26
  And as Minda was talking about, what are the evals that you're using in order to test for safety?
• 32:32
  And so again, not just the awareness of being able to do them.
• 32:36
  And then I think of the other thing that, again, there's the list, this is as long as my arm.
• 32:41
  But there's a difference.
• 32:42
  Minda made reference to it before.
• 32:44
  If you say, look, I'm going to use the NIST framework.
• 32:46
  Well, great.
• 32:48
  But there's a big difference between this PIPA
• 32:50
  piece of paper that says NIST framework and me stating that and operationalizing it, right?
• 32:56
  The execution, not just whether or not you're executing, which is a big deal, but how well you do it is it makes a huge difference, right?
• 33:05
  And that is where the learning curve matters, right?
• 33:08
  You sometimes people talk about speed as a strategy.
• 33:10
  It's not just because you're fast, but if you get there earlier, you start learning earlier.
• 33:16
  It takes time to learn how to do this and do it well.
• 33:20
  And so again, you you need to be aware, you need to know the types of things you can do, but then you need to learn and then you need to achieve scale because again, you know,
• 33:30
  again, as long as the day is, people will say, I had a successful pilot.
• 33:35
  Okay, when is the CFO going to talk about it?
• 33:38
  Right?
• 33:39
  And all of that, all of that practice, all of that discipline with getting from a successful pilot.
• 33:46
  both in terms of the AI value, but also the AI trust, transparency, interpretability, explainability, all of those things, getting that to scale are some of the things that are
• 33:58
  both hard to do.
• 33:59
  The flip side of that is when you achieve it, then you'll be running faster.
• 34:05
  So anyway, I went on a little bit there, Mehta, you should jump in too.
• 34:10
  Yeah, well, let me ask you both one last questions we're going to have to wrap up, which sort of dovetails from that.
• 34:15
  Obviously, things do change really fast in this area and there's a lot of uncertainty out there.
• 34:22
  But let's say we project, say, three years into the future from where we are today.
• 34:28
  As of today, we're what, about two years, two plus years in since since ChatGPT.
• 34:34
  So let's go about that much more time into the future.
• 34:38
  Will we be at a point where there's more clarity and stability and widespread adoption, especially in these areas of AI trust governance and so forth?
• 34:51
  Or will we still be having this discussion that it's variable, which companies are adopting and there's still uncertainty about the standards and so forth?
• 34:59
  Mehta, you first.
• 35:03
  I think my dream would be, I wish I said that there'll be 100 % clarity on entirely what trust means and everyone would be on the same page.
• 35:15
  Maybe I'm not the most optimistic on that dimension, but I think what I'm optimistic about is the attention that people will give to trust with evolving capabilities of AI.
• 35:26
  I think there are a lot of enterprises that are looking at multimodality.
• 35:31
  I had told myself that I will count the number of times I say agentic in the podcast, but I didn't say so many times, but, or deep seek, but I think with the enterprise focus on, I
• 35:45
  want to get value from the most evolved AI capabilities.
• 35:50
  The focus on governance of it from multiple stakeholders is going to rise a lot.
• 35:56
  I think the second change, which I would love to see in
• 36:00
  more of public discourse is how do you bring enterprises into the conversation around trust?
• 36:05
  So I think there are a lot of multi-stakeholder discussions that I've had the privilege of participating in.
• 36:11
  And I think there is a really good representation of large technology companies, policy think tanks, and governance bodies.
• 36:19
  I think there is room or opportunity for more enterprise, know, CLOs, CROs, CIOs.
• 36:26
  and creating space for them to understand their trust concerns and voice their innovations, I'm optimistic that that will happen more.
• 36:36
  Mike Lee gets the last word.
• 36:37
  Yeah, I mean, look, if I look at trend lines, right, you know, they're all in some ways, quote unquote, moving in the right directions, right, the technology to, you know, to to
• 36:47
  to achieve responsible AI continues to move forward.
• 36:51
  The awareness as we've been doing surveys and talking with the corporation and corporate leaders continues to increase and the actions being taken in order to implement AI trust
• 37:01
  and transparency and responsibility continue to increase.
• 37:05
  And yet.
• 37:07
  It takes a lot of effort and time.
• 37:09
  And so do I think all these problems will be solved in two or three years?
• 37:11
  I do not.
• 37:12
  I am very hopeful that more and more leaders and organizations will be taking concrete steps to make things better and to really advance forward.
• 37:22
  But I think there'll still be work to be done in two to three years.
• 37:26
  All right, well maybe we'll have to have you back on and we'll see if we take a bet, who wins the bet.
• 37:30
  But I would be on your side as well.
• 37:32
  Thank you both so much for this conversation.
• 37:34
  Thank you, Kevin.
• 37:35
  Thank you so much.
• 37:36
  This was super fun.