Latest / The Road to Accountable AI / Lara Abrash: How Organizations Can Meet the AI Challenge

Transcript

Lara Abrash: How Organizations Can Meet the AI Challenge · The Road to Accountable AI

• 0:00
  Lara, really great to have you as a guest on the road to accountable AI.
• 0:05
  Great, thanks for having me.
• 0:06
  I'm so happy to be here.
• 0:08
  There's been a tremendous amount of excitement about generative AI in particular, at least since late 2022.
• 0:15
  But recently there have been reports that companies are not necessarily seeing the payoffs they expected and questions about whether maybe the hype wave is passing.
• 0:25
  What are you seeing and what's your perspective at Deloitte?
• 0:29
  Generally, that's an accurate statement.
• 0:31
  The hype is starting to settle down.
• 0:33
  If anything, we're starting to see maybe even more impatience with seeing companies want to unlock value.
• 0:42
  some of it is there is experimentation going on all over.
• 0:46
  You'll hear examples of use cases that are.
• 0:49
  delivering, I'll say some benefits, most likely in the area of productivity, but these really large, big on lots of value, we're not seeing that happen.
• 1:01
  And there's a variety of reasons that are driving that, but that is definitely where we are in this curve, but it can and will change quickly.
• 1:10
  What's going to make a change, you think?
• 1:12
  Well, some of the issue is really just learning.
• 1:17
  so getting each individual company, each individual department trying to focus on their own use case, at this point, it's a bit like whack -a -mole.
• 1:26
  So I think part of this is starting to focus on who is going to be best positioned to identify scalable, large ways of developing use cases.
• 1:38
  That's one area.
• 1:39
  So we're starting to see a lot of tech
• 1:42
  providers consulting firms like ourselves are starting to look to industry.
• 1:48
  pervasive solutions.
• 1:49
  So there's a big area is really trying to figure out a place that we can concentrate it.
• 1:53
  The other thing is really realizing that most companies were not in the best position to really drive scalable, generative AI, whether it's lacking a data strategy that they
• 2:06
  really need to be able to do this, having the right flexible technologies, having an environment where there's a, I'll just say a desire to adopt and to move quick.
• 2:18
  focus on productivity and quite frankly, some wasted money.
• 2:22
  And that just becomes a virtuous cycle of people getting a little bit timid and afraid of this.
• 2:28
  it's going to require investment.
• 2:30
  It's going to require taking some risks.
• 2:32
  And then ultimately, it's more expensive than people probably thought it would be at the onset.
• 2:38
  This is very different from all technologies we've seen in the past where there is no day one and it's all over.
• 2:45
  The cost of keeping and maintaining
• 2:48
  models goes on for a really long time and people are starting to realize that you know building versus buying an LLM and then maintaining an LLM on their own is just more
• 2:58
  costly.
• 2:59
  So there's a lot of things coming together that are preventing sort of the day one really big unlock but that won't last forever.
• 3:07
  I this is still like an amazing technology that's going to do a lot of great things but we're not where I think CEOs and boards thought they'd be at this point in the game.
• 3:18
  Where does trustworthy or responsible AI fit into that process of scaling up?
• 3:24
  Yeah, so it's a really important part of it.
• 3:27
  It is something that I'll just say companies are all talking about.
• 3:31
  They understand there's a need to be trustworthy.
• 3:34
  There are different phases of, I'll say, maturity relative to what that would mean.
• 3:39
  And building that in on the front end is yet another thing that is preventing them from maybe moving super fast.
• 3:46
  Because not all the companies really have a framework, have the ability to have their compliance organization set up.
• 3:54
  mentioned, the data, the privacy, all those things need to be really built and thought through on the front end.
• 4:00
  And then there's a certain amount of testing of these systems to make sure they're actually creating safe and secure outcomes.
• 4:06
  So that's another important barrier that companies are all learning about.
• 4:11
  And what you're also seeing is the tech companies and consulting firms are starting to build that into their solutions early on, as opposed to, again, making every company try
• 4:21
  to do it themselves.
• 4:24
  I mean, it's obviously not something you can generalize across all the different engagements you have, but what are the major things that companies are coming to a
• 4:32
  consulting firm like yours for in this trustworthy AI area?
• 4:37
  What is around, is this something that we do separate and distinct from developing the generative AI strategy, or is it something that we should be embedding?
• 4:46
  And our philosophy is generally embed this on the front end.
• 4:51
  So when they're coming to us, it's really around strategy of what is a gen AI strategy feel like and implemented like into our overall strategy.
• 5:01
  And then how do we then get this trustworthy AI done properly?
• 5:05
  What are the steps we need to be taking?
• 5:08
  as we start to build solutions and whether it's the capabilities of the people that are around the development and deployment of the technology, how the data strategies are being
• 5:17
  built, the compliance of the regulatory environment that this particular client is in.
• 5:22
  But the key is they're looking to us to be part of it from the overall strategy, not a trustworthy AI strategy.
• 5:30
  That in itself will never get it integrated and embedded into the technology.
• 5:36
  How convinced are they that there's a need for that trustworthy piece?
• 5:40
  That it's not just something that's for show or really peripheral to the business.
• 5:46
  They all say the right thing.
• 5:47
  think depending on the company, you're getting legitimately different answers.
• 5:51
  There are companies that are global and already because of regulatory environments around the world that are different than if you're a US company.
• 6:02
  There already are requirements that they have to put in place.
• 6:06
  Then you've got sort of this regulatory overhang.
• 6:08
  Even today, there was an article about the state of California coming out with some potential legislation, which is, again, maybe contrary to what people were thinking this
• 6:17
  would go.
• 6:18
  So regulation is a big thing that I think is an overhang.
• 6:21
  The boards are asking for it.
• 6:23
  We can talk later about where the boards are generally.
• 6:26
  But if you're a tech company and you're building in a generative solution and you're not talking about trustworthy, your board, your management team, this is your brand.
• 6:36
  I mean, this is that you can have a vulnerability come to fruition after.
• 6:42
  it's real, but there are other places that are talking about it, but it's not necessarily coming with real action.
• 6:49
  And, you know, I worry that we're going to have a major footfall because if you don't get this right everywhere, you run the risk of getting it wrong everywhere because it's not
• 7:01
  something, you know, what people are going to learn about generative AI is such a powerful technology.
• 7:06
  and it's relating a lot of ecosystems coming together.
• 7:10
  One place having a significant footfall will have a major impact on the technology and how it's being, I'll say, affecting other people throughout the environment.
• 7:23
  What would you say is the biggest area of concern?
• 7:26
  There's a whole range of issues from bias to explainability to intellectual property, privacy, and so forth.
• 7:33
  Is there an area that has come to the forefront in terms of these trustworthy AI concerns?
• 7:39
  Well, definitely bias is one, because it can mean a lot of different things.
• 7:43
  A lot of people jump to bias as meaning in itself, it's just taking certain classes of people and impacting them differently.
• 7:51
  But the reality, biases could mean so many different things to how the data is formed, how it's used.
• 7:57
  So I'll just say that's an area that comes up in almost every conversation we talk about.
• 8:01
  And it's whether the data itself and how the technology is being run is actually representing the same
• 8:10
  information of the people that are affected as to what you would expect if a human would do it.
• 8:14
  Now the reality is that there's bias in human behaviors as well and so we could talk about that as well.
• 8:20
  People I think have a higher expectation for this technology than even the human being but bias is definitely the one that I would say I hear more frequently.
• 8:29
  Of course safety and security are other ones as well.
• 8:33
  know, is it creating harmful outcomes?
• 8:35
  A lot of that can come back to bias.
• 8:38
  And what is an organization that sees that problem need to do first?
• 8:44
  What's the most important thing to address those concerns about bias?
• 8:47
  I know this is going to sound really fundamental, but the first thing is actually having a framework of what their guidelines and guiding principles are around what it means to have
• 8:58
  trustworthy AI and what their objectives are for achieving an outcome that's free of bias and however they measure that.
• 9:06
  Again, it could be the same amount of bias of a human being with doing it, but starting at the very forefront, there needs to be an intent as to what does an organization want to do
• 9:16
  relative to
• 9:17
  how it builds the technology.
• 9:19
  And then it could be a variety of who's actually sitting around the table.
• 9:23
  A lot of companies have AI councils that they formed and the job of the council is really to start to think about how is our data being parsed and used and is the data actually
• 9:34
  reflecting the information that we want.
• 9:37
  And then there's all sorts of testing on the systems itself.
• 9:39
  But it does really start at the very highest level with a set of guiding principles.
• 9:45
  And that set of guiding principles should be
• 9:47
  for all the elements of what's trustworthy.
• 9:49
  And it needs to have attention to, I'll just say the commercial drive for moving AI a lot faster.
• 9:57
  And there's going to be a trade off all the time relative to the pace you can move to create something that's trustworthy, but maybe getting the impact that you want to get.
• 10:07
  Yeah, I found it interesting that there was a report that Deloitte put out on generative AI in the enterprise that talked about trust in two different senses.
• 10:15
  Trust in terms of trusting the accuracy and reliability of the outputs of the AI system, but also the employees being trusting that this is not just going to replace them or make
• 10:27
  their lives worse.
• 10:29
  Isn't there attention there, though, that the things that might be most efficient for the business might not be the things that lead in
• 10:36
  the direction that is most either in the interest of employees or in the interest of the most accurate system.
• 10:42
  Yeah, well, the first thing is just if you want to have the organization adopt these changes, there actually has to be a willingness to do it.
• 10:51
  At the end of the day, you could drag a lot of time out if the employee base ultimately does not really commit to the change.
• 10:59
  So you don't really have a choice.
• 11:01
  You do need the employee base to trust you.
• 11:04
  It's ultimately about being really honest about what you know and don't know.
• 11:08
  A lot of clients I talk to have said to me that this
• 11:12
  This is ultimately about re -skilling and up -skilling.
• 11:15
  And if you believe them, it's ultimately about taking the work that someone does today and take elements of it and say, you know, this is going to be done by the technology, which
• 11:24
  is not a novel concept.
• 11:25
  This is already happening in automation and bots and things like that.
• 11:29
  but that the individuals themselves are gonna be at a higher end of the value chain and they're gonna be doing things that they wouldn't otherwise been able to do.
• 11:38
  But the proof is in the pudding.
• 11:39
  If at the end of the day, someone says that to the entire organization and then goes and lays off everybody, well then they haven't been truthful.
• 11:47
  it is going to be, the reason we talk about it, it is going to drive the pace of change if the employees are not adopting it because the people who know the use cases,
• 11:56
  or the people that are doing the processes today, you need them to be bought into helping with the solutions.
• 12:02
  It's a circular issue.
• 12:05
  So I think companies just need to be really honest about what they know and don't know.
• 12:08
  And the reality is we may not have all the answers about what the benefits will be and what some employees' life will be after.
• 12:15
  But it will make their job a lot easier and better.
• 12:19
  And ultimately, we'll see if we can skill or upskill them.
• 12:22
  And that's why it's so important.
• 12:23
  Otherwise, you will not have an adoption of change.
• 12:26
  How should a company though think about where to draw the lines?
• 12:30
  What are the activities that it makes sense to try and outsource to the AI versus the things that you want to ensure that there's still human involvement?
• 12:38
  Yeah, I'll say I have a strong bias here that this is about making the human better.
• 12:46
  This is not about replacing the human at the end of the day.
• 12:48
  I know there's all sorts of videos going back decades about what AI could do, and the AI will build the AI, and then we won't have a need for humans.
• 12:57
  I think the reality is, as long as we all live on this earth, human beings, on one hand, need meaning and purpose.
• 13:05
  And jobs create meaning and purpose for a human being.
• 13:08
  And two, they bring things that ultimately the technology will attempt to do, but I don't believe it will ultimately ever get to the level of a human being, the creativity, the
• 13:18
  emotional quotient.
• 13:19
  So when the technology is being built, it should ultimately be thought of as how is the technology enhancing what a human can do, as opposed to how is the human enhancing the
• 13:29
  technology.
• 13:30
  And so it's really about driving it to make them better at what they do.
• 13:35
  We use the word co -pilot.
• 13:37
  The technology is the co -pilot, but I'm ultimately flying the plane.
• 13:41
  I've got the years of experience, the expertise.
• 13:44
  It creates challenges down the road relative to grooming a workforce to be able to lead a plane and have a co -pilot.
• 13:51
  Because sometimes you need to be the co -pilot to be the pilot.
• 13:53
  We could talk about that too.
• 13:55
  But ultimately, it's got to be something that makes the human perform at a higher level.
• 14:02
  and ultimately take away things that the human, it's not really adding value.
• 14:07
  It's just not an area that the human being can add value.
• 14:10
  You mentioned earlier the position of boards.
• 14:14
  I'm curious what you're hearing from boards that you talk to right now about trustworthy AI in particular.
• 14:21
  Yeah, this is an area where I would say boards are, again, in different levels of maturity, depending on the board that they sit on.
• 14:29
  Most boards are in the, if I put maybe sort of again, these tech companies or the people that are more sophisticated in this space, they're in the place of trying to understand
• 14:40
  what does it mean to actually have a board that understands generative AI?
• 14:45
  Is the CEO and the management team doing the things that they should be doing?
• 14:49
  They're more focused on the
• 14:51
  the benefits and implications of generative AI and getting themselves rallied around that.
• 14:57
  The way I see trustworthy AI comes up, they don't necessarily use these words, but a major part of their responsibility is around the reputation, the brand of the organization that
• 15:09
  they're governing.
• 15:10
  And so the focus they'll have is are there things that we need to be doing, are there things that could potentially be happening that could inhibit our brand?
• 15:18
  Now, if they're regulated back to that place, that's a core responsibility of the board.
• 15:24
  But without that, right now, when I hear it come up in a more overt way, it's really around the brand responsibility, the risk appetite of the company.
• 15:33
  Is this driving more risk?
• 15:34
  The word trustworthy AI is not used as much in boardrooms, but in essence, it's getting to the same topics.
• 15:42
  How are boards though going to get the expertise that they need to deal with these kinds of issues, especially if it's not a tech company or very technically.
• 15:53
  It's an interesting thing.
• 15:54
  I would say a few years ago there was a philosophy, I'll just say broadly about generative AI, that it was really all about having everybody be deep technologists and having
• 16:06
  everybody be deep in the understanding of generative AI.
• 16:09
  And what people have come to understand is, are we good?
• 16:14
  It said it was a...
• 16:16
  Yeah, we can start.
• 16:18
  you just go back to the top of that answer if you don't mind.
• 16:20
  Okay, so what was the question that it was about?
• 16:22
  What boards are doing?
• 16:24
  expertise of boards gaining the necessary expertise.
• 16:27
  so it's really interesting.
• 16:28
  A few years ago, people, when they were talking about generative AI broadly, not just in the boardroom, they would talk about the need to actually develop deep technologists,
• 16:39
  people who had really, really deep understanding of generative AI, technology, and data.
• 16:44
  And now if you fast forward to today, the reality is there's much less of a focus on that.
• 16:50
  And it's really around the best athlete approach.
• 16:53
  And what we want are people that have a domain expertise that know enough about the technology to know the implications.
• 16:59
  And I would say you see the same thing in the boardroom.
• 17:02
  The more sophisticated companies are having a, they may have technologists in their boardroom, but it's not necessarily to create a 12 or eight person technology board.
• 17:13
  It's to have some people in the room that may be super deep.
• 17:17
  For the others, they're focused on fluency and creating enough of an understanding.
• 17:22
  but without losing the domain expertise.
• 17:25
  What makes them effective in the boardroom is their skills around governance, their skills around running businesses, and then ultimately the industry and the company they're
• 17:35
  governing, knowing it so deeply that they can now question and probe those things.
• 17:40
  The other thing I'm seeing and I'm a big supporter of is sort of on -demand learning, making sure that when in the life cycle it makes sense to create a learning environment
• 17:51
  into the boardroom.
• 17:52
  they're doing that, whether it's bringing in experts, law firms, consulting firms, technology companies to bring an outside objective voice, someone other than the
• 18:01
  management team sort of articulating, this is what's going on in the outside world, or finding other sort of educational things they need to do.
• 18:09
  So I see a mosaic in the boardroom.
• 18:13
  It's not eight or 12 technologists.
• 18:15
  It's people that really understand the role of governance, business, and the client the companies are working with.
• 18:22
  but that they're sufficiently steep in this technology that they can ask the right questions.
• 18:28
  With generative AI though, it's interesting in that on the one hand, this is very deep, very sophisticated, very complicated frontier technology.
• 18:37
  On the other hand, there's this magical experience that you have that you get the interface and you talk to it in natural language and so forth, which can to some extent
• 18:46
  create the illusion that it's simpler than it is or that it really is a person on the other end.
• 18:51
  is having broad understanding of what's going on beneath the hood
• 18:58
  more important in this environment or less so than with the previous waves of AI and tech development?
• 19:03
  definitely more important.
• 19:05
  doesn't, again, it doesn't mean it's just a GEN -AI board, but you do need to understand the example I gave you earlier that this is not a six or nine or 12 month sort of
• 19:16
  technology implementation and then it's off to the races or some standardized.
• 19:22
  off the shelf package that you're using.
• 19:24
  This is ultimately something that's a breathing technology that could do it.
• 19:28
  It could go off.
• 19:29
  It could have hallucinations.
• 19:31
  And you ultimately need to understand the risks of that from a broad, the board perspective.
• 19:35
  And I would say the thing that probably makes this, in my opinion, much more board worthy is there's immense opportunities, but there's also immense risks to business models.
• 19:47
  than other technologies maybe have likely done where parts of a business may be disintermediated or ultimately a new entrant can come in because they can access things
• 19:58
  like generative AI and create a different type of workforce.
• 20:01
  So the role of the board ultimately needs to be the long -term value of an organization.
• 20:07
  And so they need this deeper understanding to really, again, objectively challenge
• 20:14
  What's actually happening with our competitive set?
• 20:16
  Be able to sense real time.
• 20:18
  Are there things that are going to suggest that what we do?
• 20:22
  We made widgets yesterday.
• 20:23
  Are we still able to make widgets in a post -GNI world?
• 20:27
  And really ask those questions.
• 20:28
  So to your point, they do need to be deeper than just implementing a SaaS technology for payroll.
• 20:36
  I mean, a lot of this is at the heart, the core of the business.
• 20:41
  You alluded earlier to the possibility that there would be a big foot fault or some big foot faults around AI.
• 20:50
  Do companies need to think any differently than they already are about risk management to anticipate and mitigate those challenges?
• 20:59
  Or is it something that if companies have a good risk program, then they're in the right position?
• 21:05
  I don't know of any company that would say that their risk program doesn't need to be improved.
• 21:10
  I've seen a lot.
• 21:11
  are some that are alcohol, black belt, and they're really good.
• 21:16
  But for all of them, this changes the risk profile of a company.
• 21:21
  It changes the type of workforce that the company is potentially going to have.
• 21:26
  It's going to require different sorts of compliance and regulatory exposures, potentially different lawsuits.
• 21:33
  coming from people coming in and saying that some technology, particularly in the early years, harmed them in a certain way.
• 21:40
  So all those things are things that are gonna suggest there's new emerging known and unknown risks that probably haven't been vetted in the past.
• 21:49
  And again, you need people with complete expertise in risk to then know enough about what this technology is doing and also being sensing.
• 21:58
  That's a big part of...
• 22:00
  I'll say a change in how the board needs to be thinking about its role.
• 22:04
  I think in the past, this idea of being so close to changes in the market weren't as relevant.
• 22:10
  If you're going to be managing and mitigating risk, you can't be risk adverse in this environment.
• 22:14
  We don't have a choice.
• 22:15
  You won't be competitive anymore.
• 22:17
  But at the same time, you need to be thinking about how do I prepare for a risk when it does happen?
• 22:24
  you know, are there insurance companies, there thick captives?
• 22:27
  Like, what am I going to do if at the end of the day I get sued by my workforce because how they get impacted by this unions?
• 22:34
  All those things are things they need to be thinking about, which is a broad topic that we could talk about today if you'd like, Kevin.
• 22:41
  I think one of the things I'm not seeing happen as much is really shaping external stakeholders.
• 22:47
  You know, there's this pressure right now to have
• 22:51
  generative AI deliver these immediate results and investors are looking for it.
• 22:56
  You have people that are in and out of these companies stocks really quickly and if they don't see it, they beat them senseless.
• 23:03
  But do the investors themselves understand what it means to have trustworthy AI and what, one and how is a company making those choices?
• 23:10
  That's just an example of there are a lot of constituents that need to be educated by a company around what it's doing relative to how it's embedding gen AI and the choices it's
• 23:20
  making.
• 23:22
  How do companies do that effectively?
• 23:25
  Well, a couple things.
• 23:27
  mean, one is just starting by, I'll say old fashioned, coming up with a stakeholder matrix.
• 23:32
  Like who are the key constituents that they're going to need to deal with?
• 23:36
  And then ultimately deciding, is this something that they do as an individual or are they better served doing it as a cohort?
• 23:43
  So if it's an industry, as an example, take life sciences.
• 23:46
  Life sciences is a regulated business.
• 23:49
  Do they come together and talk about how they think they're applying the
• 23:54
  regulatory requirements already to what they're doing, know, should they be subject to even more requirements?
• 24:00
  How do they take investors who invest speculatively across this industry?
• 24:06
  Do they use trade groups to talk through this?
• 24:09
  Do they, you know, hire like law firms to go out and then they can gauge and lobbying effects to basically start to bring them along the way?
• 24:17
  But until you do that, you've got a group of people.
• 24:20
  And I'm using investors as an example.
• 24:22
  You could use credit agencies, other shareholders, the employee group.
• 24:26
  The list will go on and on.
• 24:28
  They need to be thinking about what they want to say to them.
• 24:31
  But it all starts with actually, I don't think we've talked about this.
• 24:34
  You actually need an integrated strategy that says how GenAI is going to be used in the organization, not a GenAI strategy.
• 24:42
  That's probably the one.
• 24:43
  I'll say major thing I hear people talk about is they need a gen AI strategy.
• 24:47
  The reality is you have a strategy.
• 24:49
  How is gen AI going to augment what you do in the execution of your
• 24:55
  So that sounds like it means that it'll vary a lot by organization.
• 24:59
  It should.
• 25:00
  mean, depending on what the opportunities are.
• 25:03
  In some organizations, this could merely be about back office productivity gains and doing things differently, leveraging GenAI.
• 25:12
  In other organizations, could be, take life sciences.
• 25:15
  How a pharmaceutical drug is now developed is completely different.
• 25:20
  The whole R &D experience is completely different because of how generative AI is being used.
• 25:26
  So once it starts impacting both the
• 25:28
  I'll say the core business as well as the back office, it has a much more pervasive effect across the organization.
• 25:35
  your sense about the way the regulatory environment is developing around the world?
• 25:39
  Well, it's, mean, outside the US, it's, we clearly have a less litigious environment.
• 25:46
  So there's a lot of reliance on regulation and that already exists.
• 25:50
  We've seen it in almost every, every topic you could speak of.
• 25:54
  Here in the US, you know, it is interesting seeing now some states start to.
• 25:59
  to focus on it.
• 26:00
  Many would say, as I mentioned earlier, that industries, many industries are already regulated and a lot of the elements that are conserved around generative AI fall under
• 26:10
  that.
• 26:12
  And then you're seeing all of these executive actions.
• 26:15
  It seems to me that there's a bit of swirl.
• 26:18
  And I think ultimately the test of this will be, does this make the US not competitive anymore?
• 26:27
  And I think if that concern gets to a certain point that we're now going to lose pace with some parts of the globe, that's when you're going to likely see pressure.
• 26:36
  But it's like nothing else.
• 26:38
  It's going to have a lot of political debates going on.
• 26:43
  Like the pandemic, the states are going to want to have their say.
• 26:46
  And you're already seeing this with California.
• 26:48
  And you've seen this with other rules.
• 26:50
  there's no clear answer for us.
• 26:51
  But I think the thing that everybody's going to be focused on, the element of concern.
• 26:56
  being competitive.
• 26:57
  And if that all of a sudden puts us at a significant disadvantage, I think that will put lot of pressure on how far can we hold our companies at hostage.
• 27:09
  I'm curious about how you and your organization address the challenges of generative AI.
• 27:16
  How do you ensure that your people have those skills they need and are in position for that use of AI to accentuate people as you talk to
• 27:27
  Yeah, so one of the first things we did, and it took a while to actually get ourselves comfortable that we could do it at scale, is we really started to create, I'll just say,
• 27:38
  areas for our people to play.
• 27:41
  And allowing them the ability to essentially go in and ideate and incubate with our own technology, things that we're comfortable with them using that meet our standards, not
• 27:53
  necessarily some third party.
• 27:55
  and allowing them the ability to start to get comfortable with it and start to create their own ideas about what they can do with it and create a, I'll call it a clearinghouse
• 28:05
  of ways to make sure we're not doing things externally that we're not comfortable with.
• 28:09
  But just the ability to have a place to play and starting to stimulate that that's actually something we want our people to do, it was a big part of getting people starting,
• 28:20
  getting them to be more comfortable.
• 28:22
  We're still the same place everyone else is.
• 28:25
  We've had some episodic outcomes where I would say a really good use case that makes a lot of sense and that people are excited by.
• 28:34
  But we haven't seen something that all of a sudden does everything we do so differently across the line.
• 28:39
  But for our people, it's just talking about it as a comfort level.
• 28:43
  We do benefit the tenure of our organizations on the younger side.
• 28:49
  this is one of the things I think over time is going to
• 28:52
  change this, I'll just say trepidation.
• 28:54
  I think about if you're a junior, junior high or high school right now, you're leveraging generative AI and technology freely, and you're gonna enter a workforce expecting this.
• 29:06
  And so we have a younger workforce, so we don't have as much, I'll just say resistance from that all of the workforce.
• 29:14
  I think broadly, all companies will start to benefit from.
• 29:18
  this push, it's going to be a natural push as people join organizations.
• 29:23
  Almost like, why aren't you using this?
• 29:25
  I used it every day in my college life.
• 29:27
  You should be using it here.
• 29:29
  So we're going to have benefit of like a bit of a passage of time that we're going to have natives coming into all companies.
• 29:37
  presumably there are some things that people use consulting firms and other professional services firms for that will no longer need to be outsourced or they won't need to be
• 29:49
  willing to pay the same amount with the what the tools provide with generative AI.
• 29:53
  So how do you think about the potential impacts on your business?
• 29:56
  Yeah, well, we definitely see some things that when we do something more efficiently, will someone say, well, is it the same pricing that you had before?
• 30:05
  So we are looking at that.
• 30:07
  We're also looking at net new opportunities.
• 30:10
  I mean, the area that I would say forget even just the latest firm, the professional services firms, the consultancy firms across the board, the big and the small have really
• 30:21
  spent a lot of time in the last decade to 15 years helping companies.
• 30:26
  grapple with technology and how to implement and run technologies.
• 30:31
  This is going to have the same, I'll say halo effect for the companies that need that assistance.
• 30:38
  They'll look to places like Deloitte, they'll look to some hyperscalers because at some point that expertise is better served outside your organization.
• 30:49
  But what the person outside doesn't know that you know better than anybody is
• 30:53
  is your own organization.
• 30:55
  it's this marriage of both, but I would say we've seen this across the board in almost every wave of technology, the last wave being around the cloud, which has had a pretty
• 31:07
  long run.
• 31:09
  All of that came to companies coming to firms like Deloitte and others to get that expertise, get that assistance, and even run some of these technologies for them as
• 31:19
  opposed to pay their own people to do that.
• 31:22
  It's going to go in both directions, I think, for us.
• 31:24
  But at the end of the day, our role is really about going where the market goes.
• 31:28
  And the market is likely helping clients think about how this technology gets deployed.
• 31:34
  So last thing, looking to the next few years, what's the biggest development that someone should look to in an organization that's going to impact development of generative AI, in
• 31:47
  particular with regard to these concerns about trustworthy AI?
• 31:51
  I think inside an organization, the work that companies are doing right now around, I'll say, really focusing on the enablement across the board and building in at the front end
• 32:05
  what it means to be trustworthy AI, that's going to be the biggest unlock.
• 32:10
  So some companies, this is the first time you're hearing the words trustworthy AI.
• 32:14
  Other organizations are hearing it and starting to think about what does this mean to build it in upfront?
• 32:20
  And without
• 32:21
  regulation or without a footfall of any sort, it's going to be this pressure of, can I do something and meet the market?
• 32:29
  And in the next two or three years, I think it's going to be ultimately their ability to really pull together that framework that I mentioned and the ability to start locking in
• 32:41
  real examples of what being trustworthy looks like.
• 32:44
  So when they have a use case that they are actually ready to pile it and go live into their
• 32:50
  into their life system.
• 32:52
  Have they done everything that their framework would suggest that they feel really good about relative to this, know, transparency, the safety, the biases, all those types of
• 33:02
  things.
• 33:03
  I think those building upon each other, people will start to realize what it means to be trustworthy.
• 33:09
  The worries I have are the foot faults, regulatory environment pushing, and even the employees really resisting it.
• 33:17
  Will all those things are gonna cause people to...
• 33:21
  their feet.
• 33:23
  That's interesting.
• 33:23
  You focused really on internal expertise in the organization as opposed to some new development in technology.
• 33:31
  Well, you could, you mentioned inside.
• 33:34
  Outside, I think ultimately what we're starting to see a trend of is companies, just like almost any other business challenge saying, do I need to do this all myself?
• 33:45
  And so they're starting to look to third party hyperscalers and to our consulting firms and saying, is this something you could do for me?
• 33:53
  So if you look at the big hyperscalers now, the idea of having trustworthy AI is part of their brand.
• 34:00
  And if they can get some big unlocks in use cases in focusing on either industry specific things or I'll say pervasive generic areas of an organization and the organization says,
• 34:12
  wait, I can go to somebody who has a built out use case.
• 34:17
  It's more cost effective and they're taking responsibility for the trustworthiness of the data and all the outcomes.
• 34:24
  That to me is the likely big unlock when it comes to trustworthy AI.
• 34:30
  Other than that you're bread and butter, for the average company it's going to be hard to invest all it needs to do to get that outcome, which is why I think the consulting firms
• 34:39
  and the technology firms, the lawyers, all these folks are going to be really well positioned to help companies think about how they make that next big push.
• 34:48
  Laura, thanks so much for having the conversation with me.
• 34:51
  All right, thank you, Kevin.
• 34:52
  Really appreciated you having me.
• 34:54
  So thank you.