Latest / The Road to Accountable AI / Uthman Ali: Responsible AI in a Safety Culture

Transcript

Uthman Ali: Responsible AI in a Safety Culture · The Road to Accountable AI

• 0:09
  Hi, I'm Kevin Werbach, professor of legal studies and business ethics at the Wharton School of the University of Pennsylvania.
• 0:15
  For decades, I studied emerging technologies from broadband to blockchain.
• 0:20
  Today, AI is promising to transform our world.
• 0:24
  But AI needs accountability, mechanisms to ensure it's developed and deployed in responsible, safe, and trustworthy ways.
• 0:32
  On this podcast, I speak with the experts leading the charge for accountable AI.
• 0:40
  I often say that while every company is going to have to think about AI ethics and governance, the way they do so will depend greatly on the nature of the organization, its
• 0:51
  scale, its industry, its culture, and so forth.
• 0:54
  That really came out in the conversation that I had with Uthman Ali.
• 0:58
  He is the global responsible AI officer at BP and also advises the World Economic Forum's AI Governance Alliance and the British Standards Institute.
• 1:08
  He's an advisory board member to OxAfrica.
• 1:11
  and Oxford University AI ethics spinoff.
• 1:13
  In our conversation, Uthman has great perspectives on distinctive aspects of doing responsible AI at a major global energy company, as well as the practicalities of setting
• 1:25
  up a program, making the case for AI governance and responsible AI, how to create an effective AI ethics committee, and what success means, with a little bit of robot ethics
• 1:35
  thrown in.
• 1:37
  Uthman, welcome to the Road to Accountable AI.
• 1:40
  Thank you for having me.
• 1:41
  You were one of the first, if not the first responsible AI lead executive at an energy company.
• 1:50
  How is it different, do you think, doing this kind of work at a place like BP versus a company in, say, financial services or tech?
• 2:00
  I I'm the first, think I'll claim it until I find someone else that might have secretly been doing this for years without anyone knowing.
• 2:07
  But I think the difference for us is that we're such a massive organization, but we do so many different things as a company, whether it's more customer facing stuff or just
• 2:15
  hardcore engineering.
• 2:17
  But when you're comparing it to financial services, what's interesting is that a lot of our systems might not necessarily be making predictions about people, they have safety
• 2:25
  implications as well.
• 2:27
  So when it comes to safety and just good engineering design practices.
• 2:30
  That's one of the most interesting things about working here, I'd say.
• 2:35
  And presumably safety is something that a company like DB has a lot of prior expertise on.
• 2:40
  So does that translate over in terms of how you and the company think about it in the air?
• 2:47
  Yeah, cause even I was thinking about this recently, right?
• 2:50
  We don't really say responsible aviation.
• 2:53
  oh The implicate fat is just part of what the product should be doing.
• 2:59
  You just expect if you're flying commercial flight, it should be safe.
• 3:04
  So, you know, it's the question of like, we need specific roles dedicated to this?
• 3:10
  And the answer to that in the short term is yes, because you need to build that maturity.
• 3:14
  And as you look at how do you actually embed.
• 3:17
  Responsible practices within existing workflows and mitigate risks.
• 3:21
  need to be doing that.
• 3:22
  But eventually you will get to a point where this is not just responsible AI, it's just AI.
• 3:28
  And a lot of the work that we're doing right now in terms of like bias mitigation or thinking about when do you do use a black box and more explainable model.
• 3:37
  Eventually you reach maturity where it's kind of expected competence that this should be being done across the board.
• 3:46
  A lot of organizations, though, see that as just something that is an additional drag on deployment and innovation.
• 3:56
  I would like to think that you're right, that all companies reach that point of seeing this as being essential to actually doing development and deployment of AI.
• 4:03
  But is it a challenge to make that case ever internally?
• 4:09
  I think for us we're able to make the case pretty strongly because it is what we do as a company, right?
• 4:14
  Safety related stuff, it's pretty essential and we already operate in a super regulated environment.
• 4:19
  So we kind of don't have a choice and it makes sense for us as a business because it is what we do.
• 4:23
  But to your point about like lots of companies, I think fundamentally just misunderstand what responsible AI is or what AI ethics is.
• 4:31
  A lot of them I think still think it's this vague fluffy tech for good type thing.
• 4:35
  And without it, it's just basically building good products and so on.
• 4:39
  So for example, I use this right like if you have a healthcare AI chatbot that's doing like diagnostic predictions.
• 4:47
  Let's say you didn't do any ethical assessment on this at all or only like customer centric thinking at all right and your development team says this has like a 93 % accuracy
• 4:57
  rate.
• 4:58
  You might say like the mind look great on the slide deck but the implication you're saying is that seven out of a hundred times this thing might be getting something wrong?
• 5:07
  Even when you're looking at development metrics and best practices, how are you even coming up with thresholds or even do technical reviews and making these decisions without
• 5:16
  first thinking by your customer?
• 5:18
  And by part of that is what are the ethical implications?
• 5:21
  Cause if the thing isn't being used well, or if it's constantly giving incorrect predictions that people are over relying on, it's not really solving a problem.
• 5:30
  It's actually just creating a burden, right?
• 5:34
  How far along are we to having the practices and standards and tools to ask those questions effectively?
• 5:46
  I would say the window of opportunity now is probably in the next 18 months to two years to really go about having a comprehensive AI strategy where you actually just bake these
• 5:56
  things in.
• 5:57
  Because right now the EU is coming out with a new standards from Cincinnati like there's the high risk AI system standards.
• 6:02
  There's the ISO standards.
• 6:04
  Every regulator across the world is looking at, we need AI specific regulation now?
• 6:09
  What are we meant to be doing?
• 6:10
  Do we uplift existing compliance requirements?
• 6:14
  So whilst this is happening,
• 6:16
  This is the golden opportunity for the companies to think what is our actual AI strategy or plan?
• 6:21
  Which regulations do we comply with?
• 6:23
  Do we take the AI actors as our gold standard across the board?
• 6:27
  Do we take a more federated approach?
• 6:29
  But in order to do that, you need to at least have a system of tracking how systems get built and deployed within the company or even procured.
• 6:37
  And when you configure that out, you can figure out which actual guardrails do we need for specific use cases.
• 6:45
  Isn't it hard though at this point, given all of that uncertainty about how regulation is going to develop?
• 6:51
  And for example, you know, the United States under the Trump administration is taking a very different approach, a much more skeptical approach to regulation.
• 6:59
  So from a company standpoint, it might be challenging to decide in this 18 to 24 month window, how we're going to move forward.
• 7:10
  I there is a risk that sort of people go around under these talking circles with the responsible AI.
• 7:17
  Which regulation do we go with?
• 7:19
  China's approach versus the US versus the EU.
• 7:22
  But if you're just really pragmatic about this, the EU is such a huge market.
• 7:26
  Eventually for most companies, they'll need to be selling stuff to it, replacing things onto the market, right?
• 7:31
  So you might start there and say, this is the strictest one in terms of requirements.
• 7:36
  What do we need to do to actually be compliant here?
• 7:39
  Like what does the documentation need to do?
• 7:41
  Even sending things for like third party audits, like conformity assessments.
• 7:45
  What do we actually need to do to be compliant with this one to begin with?
• 7:49
  When you figure out the most difficult one, you can look at other religions and say, actually, this is overly burdensome.
• 7:54
  We don't need to be doing this.
• 7:55
  We don't feel as a company to even protect our customers or our reputation.
• 8:00
  But it's only when you figure out the baseline of the hardest one, you can make that informed decision.
• 8:07
  Got it.
• 8:07
  And if a company doesn't have a responsible AI or AI governance program, where do they start given how many different elements there are now in this area?
• 8:19
  I say in terms of where to begin, the easiest thing would be to be in, at which fields are most adjacent within the company that which departments already exist, right?
• 8:31
  Before even looking at do we hire someone external or build a whole new team?
• 8:34
  You've got to look at like, what do we actually have right now?
• 8:37
  So companies would probably look to date digital security or data privacy.
• 8:41
  I was getting step one to say like, look for digital compliance.
• 8:45
  What do we have right now?
• 8:46
  What actually happens?
• 8:47
  What's the processes?
• 8:48
  Either even joined up or they're parallel sort of approval streams.
• 8:52
  Then you can look at actually, do we even have an internal AI team?
• 8:55
  Do we have internal data scientists?
• 8:57
  Do we just buy everything off the shelf?
• 8:59
  Do we rely on third party contractors?
• 9:02
  But when you have those, you can figure out, okay, even how many systems do we have?
• 9:06
  Like even when you look at actually, these might be fragmented inventories.
• 9:10
  Companies might not realize how many high risk AI systems actually have in the company until they basically do.
• 9:17
  essential housekeeping.
• 9:19
  So that's the starting point for us doing that inventory.
• 9:23
  Yeah, the inventory, it's, it's one of the most important ones.
• 9:26
  that's because we split governance in two ways, right?
• 9:30
  There's what responsible AI is, which we think of like ethics, legal and compliance.
• 9:36
  But then there's just good AI governance, which is the commercial governance, which is, we actually making money from this?
• 9:42
  Is it delivering value?
• 9:43
  Do people actually like the stuff that we're building?
• 9:46
  And a lot of this will start from the same point, though you need a system of tracking all the AI stuff you're doing in the company, or how your employees are using it.
• 9:53
  Got it.
• 9:54
  How did you personally get engaged in this work?
• 9:58
  Cause when I was studying the law, thought this is really, really boring.
• 10:02
  I didn't want to do, I really just did not want to do traditional corporate legal work.
• 10:07
  But I was always interested in human rights, but particularly around the, and it's something that really fascinated me was robot ethics and the ethics of what humans really
• 10:15
  calls.
• 10:15
  Cause I thought this was one that if you can figure out how to do AV safely, you could largely get most other technologies done safely too, because
• 10:25
  There's certainly interesting ethical issues or the privacy surveillance, even anthropomorphism of people even humanize the technology, safety issues, even like just
• 10:33
  getting incorrect predictions, leading to accidents, job displacements.
• 10:37
  thought that was like a really interesting case study.
• 10:41
  look at what can you, if you can figure out your governance around this, most of the technologies that are super complicated, you can figure out how to govern them as well.
• 10:50
  But what I found was a lot of traditional lawyers.
• 10:53
  really did not like talking about digital ethics at all, which I found interesting because they said, look, it's really complicated.
• 10:59
  It's a gray area.
• 11:00
  There's no real right or wrong with lot of this stuff because there's no existing case law, right?
• 11:05
  It's still being formed.
• 11:06
  So what I found quite similarly, engineers and lawyers often think quite procedurally.
• 11:12
  It's sort of what's the flow chart, what's the process, how do I think about this?
• 11:16
  If then, except lawyers often do it with words.
• 11:20
  Engineers are just flow charts and diagrams.
• 11:23
  And part of the job of being like a global response by officer is taking the critical thinking out of people's hands with an expert committee to help actually guide them
• 11:31
  through what is the right decision on the company's behalf.
• 11:35
  So essentially taking it and turning it into something that can be implemented in a flowchart.
• 11:40
  Kind of yeah, flowcharts unfortunately is a big part of the job.
• 11:43
  Cause you got to figure out the flow.
• 11:45
  We joke about it so much, but so much of it comes down to like just the flow charts of internally, how do things work?
• 11:51
  And engineers love flow charts.
• 11:53
  And if you can explain things in flow charts, for some reason the penny drops.
• 11:58
  Have you just sort of get things?
• 11:59
  So you were like first doing this years ago, I just couldn't understand how people are wrapping their heads around it.
• 12:04
  When you explain like even how governance processes work internally in the flowchart, people be like, oh.
• 12:09
  That makes sense.
• 12:11
  But for me, because as I have it from a legal background, you're trained on words and text.
• 12:15
  It made no sense to me, but for them, I was like, look, you give your customer what they want.
• 12:19
  They like the flow charts and diagrams.
• 12:22
  So that's how they like even ethical guidance, things to be done diagrammatically as opposed to long blocks of text.
• 12:28
  That's really interesting.
• 12:29
  That's the second half after the issues have been reduced to uh procedures and requirements.
• 12:37
  How does a company address those hard ethical questions at the front end where there's not an obvious right or wrong answer?
• 12:43
  This is obviously one the most difficult things for companies to come up to because when it comes to even issues like bias and fairness, right?
• 12:49
  Or even is it the right thing to be doing this or not?
• 12:51
  The question often turns to who's going to make that call within the company.
• 12:54
  So the first step is to look at like, what is your risk management approach?
• 12:58
  In other areas, even outside of digital ethics or AI ethics, there already should be ethics based guidance or the procedures how to do this, even with just physical systems or
• 13:07
  even when it comes to just engineering stuff.
• 13:10
  So there's already things you can tag into.
• 13:13
  doesn't necessarily need to be reinventing the wheel.
• 13:16
  But to do it effectively and scalably, the best thing you can get is top-down sponsorship to create the expert committee of the interdisciplinary skills to basically create that
• 13:25
  ethical pace law to say like, look, when it comes to digital assistants and co-pilots, this is how we're going to deploy this across the company.
• 13:33
  This is the company's basically approved stance.
• 13:35
  These are the people to speak to if you need guidance, because what you don't want is seven different teams in these huge organizations.
• 13:42
  each having their own interpretation of what ethical AI is and just sort of doing it and it becomes very patchwork and it becomes a bit of a mess with internal inconsistencies.
• 13:50
  And for this expert committee, one of the key things is that it's not really about just your own personal beliefs and views.
• 13:57
  You have to look at what are the company statements around this, even externally or internally, because it has to be based around something like even the company code of
• 14:04
  conduct, sustainability aims, ESG, human rights policies.
• 14:09
  There's a lot of stuff
• 14:10
  that these large organisations have already signed up to that you sort of need to harmonise.
• 14:16
  Because the ethical guidance you give can't just be, is just my opinion, off the cuff.
• 14:21
  It needs to be rooted in the company's sort of ethical stance that makes sense and is just coherent.
• 14:27
  Many companies have these AI ethics committees or hearing committees or governance committees.
• 14:31
  um What do you think is most important to build them in ways that they actually are effective and can reach that kind of uh convergence that you describe where it's not just
• 14:42
  a group of opinions?
• 14:44
  But the first thing you need to do is probably start with a top-down sponsorship over what is the committee actually going to be doing.
• 14:51
  Because even approaching C-level, that's the chance for everyone to write down what is the point of like this committee and is it, need one committee, does each business unit have
• 15:01
  their own committee?
• 15:02
  But before approaching one of the C-level executives, it forces everyone to actually write down what is the plan?
• 15:08
  What is the purpose of this?
• 15:09
  How are we going to measure success?
• 15:11
  Because committee to me might be something different to you.
• 15:14
  Committee to someone might be board, right?
• 15:16
  Some sort of governance board mechanism.
• 15:18
  Committee to others might be some advisory committee that gives recommendations, but they don't have teeth.
• 15:23
  So that really needs to be ironed out because they're often working responsibly.
• 15:29
  We use lots of similar words, but they mean different things to different people.
• 15:33
  And if you have a committee, having people in meetings talking about something doesn't solve the problem.
• 15:39
  If they don't have the skills, you've got to look at what skills do you actually need?
• 15:43
  For many companies, what they'll be missing is an actual ethicist.
• 15:47
  Someone who is skilled in negotiations and making trade-offs that has application experience with AI.
• 15:53
  That's often the missing component because they might have lawyers, have cybersecurity professionals risk management people, but they probably will struggle to find dedicated
• 16:03
  professional ethicists that have AI experience.
• 16:07
  And the committee as well, one of the risks I often see is that
• 16:11
  They're sort of left to the risk management side of the company without people on the innovation side.
• 16:16
  You don't want a group of people coming up with all the reasons why you shouldn't do something without someone saying, this is why we should be doing it.
• 16:24
  Right.
• 16:25
  It needs to be a balanced approach.
• 16:27
  Let me get back to your own experience.
• 16:29
  You talked about your interest in robot ethics and some of these questions.
• 16:34
  At BP, did you make the case for creating a responsible AI role or was the company was already going to do and they found you?
• 16:43
  Interestingly, when I have joined, it was originally as a digital ethics lead within the research team, because the company is doing a huge transformation, investing a lot in
• 16:50
  digital AI and other technologies too.
• 16:53
  But they said we need an ethics specialist because lots of the dilemmas they say, even people concerned with like job role changes or what does this mean for the future of work.
• 17:02
  Like we need someone here that can basically teach us what is digital ethics, but work with some leading universities.
• 17:08
  And what happened was I worked with the university of Oxford on a research paper called CAPAI, which was a detailed conformity assessment procedure for the EU AI Act.
• 17:18
  And people were saying AI Act's being negotiated right now.
• 17:21
  People are talking about it.
• 17:23
  How do you actually do this?
• 17:24
  Like what does ethics look like from the start of a project all the way to the end?
• 17:28
  So working with Oxford Business School, they developed a protocol for testing projects.
• 17:33
  And we worked with them and several other companies.
• 17:35
  Now they actually create an AI governance startup out of this called Moxethica, which sells AI governance tools, which is a risk assessment platform.
• 17:42
  But out of doing this work, it be having a legal background, but also trained in ethics and also knowing just being embedded within the technology teams.
• 17:49
  was just a natural progression.
• 17:51
  So we need a dedicated role to this to be specifically looking at AI.
• 17:55
  Given that with post-gen AI, mean the world changed, right?
• 17:58
  This went from being a side thing, which may be important.
• 18:01
  We know it's so boring but we've got some time to...
• 18:03
  We don't have time anymore, we need to do enough.
• 18:05
  We need to fast track another thing that we were doing.
• 18:09
  Right, well, clearly companies had to fast track their initiatives in this area, but it's interesting that the D.P.
• 18:15
  also felt like this was the time to fast track the ethics and responsibility.
• 18:20
  think for us it was such an important strategic decision for the company because there's also clarity over again, what is our approach to AI going to be in the future?
• 18:29
  But one thing that I really liked about even having this role at the company was sort of planting a flag in the sand that this is an area we're going to walk back on.
• 18:38
  This is part of the company's future now.
• 18:40
  It's cemented in the company's legacy.
• 18:42
  Would be the first one to even create this role or start building out this department.
• 18:47
  And I think that again, like for us, the BP was foresight, like even me had been hired into the R &D team.
• 18:53
  It was more a gut feeling or acknowledgement that in the future, this is going to be super important for us as a company.
• 18:59
  But even creating this role now for Responsible AI and the stuff that we're doing right now, five years from now, the company, you would hope that people working here would
• 19:07
  really benefit from the stuff that we're doing today.
• 19:09
  What's the hardest thing in terms of either creating or implementing this kind of program?
• 19:17
  hardest thing is probably the size of the company of ours.
• 19:21
  It's like when you're looking at like creating the program and embedding it, it's often where do you start?
• 19:27
  Because even creating the AI inventory, we've got many different departments across the company, different business units, and you really have to learn how does the company work
• 19:35
  and how the different departments work to actually make this effective and create a frictionless process for projects to be screened without people thinking this is
• 19:43
  additional red tape.
• 19:45
  So a lot of it was actually understanding what are our current even governance approval streams?
• 19:50
  How can we consolidate all this?
• 19:52
  If there's any duplication hack, we just make a better customer experience and approaching it to like the customer first lens, like our employees are our customers.
• 20:00
  If we're asking them to send projects for approval, we'll ask them to do ethics based technical reviews.
• 20:04
  Where do you go?
• 20:05
  How can they make this as simple as possible?
• 20:08
  And it was also just getting a lot of buying from different departments.
• 20:11
  So everyone is clear on what we're trying to govern.
• 20:14
  Because when you say AI governance to the commercial side of the AI team, they're thinking about reusable components.
• 20:20
  How do we build better code?
• 20:22
  How do we make sure projects delivering value speak to the legal team?
• 20:26
  Like, aren't we legally compliant?
• 20:27
  Then ethics is actually a bit of in between.
• 20:29
  It's protecting the reputation, but building better stuff, right?
• 20:33
  Digital security again, is preventing cyber risks.
• 20:36
  So if you even be clear again on the language of this is what good looks like for all of us.
• 20:41
  How do we create a process or program here that basically delivers everything that we need?
• 20:46
  Where do we converge or when there are differences, where do we diverge?
• 20:51
  And in the context of generative AI, how do you deal with the fact that it may be impossible to be certain that there's no such thing as a fully unbiased system, know,
• 21:03
  pollucination is a uh somewhat inherent risk, although it can be mitigated.
• 21:07
  How do you think about the fact that you can have a very good program and yet, you know, there might still be issues?
• 21:14
  I think the pragmatic approach is that there will always be issues but not all unintended consequences are just unforeseeable.
• 21:22
  For example, if you actually think through, okay, this Gen.
• 21:25
  chatbot might produce some inaccurate outputs.
• 21:27
  Before we go deploying this to 10,000 people, let's just actually think for these use cases, what could go wrong?
• 21:35
  Which bits of the business need targeted training or do we say this use case is inappropriate, for example, or do we say look,
• 21:41
  use a traditional machine learning method where there isn't that hallucination rate.
• 21:45
  One of the most effective ways to even deal with this is also just recognize that within each different department of the company of our size, how the risks manifest or what those
• 21:54
  consequences are, are vastly different.
• 21:56
  If you're dealing with a team in marketing and you've got AI producing inaccurate outputs and you give someone the classic like extra-fingered type thing, that can be ethically
• 22:06
  embarrassing or reputationally bad and you might need due diligence if you need them.
• 22:10
  before anything gets put external, for example.
• 22:13
  But if that's being used in safety critical context, that same risk of using the technology is manifested very differently.
• 22:21
  The harm is completely different.
• 22:23
  So you just could be very pragmatic and understand the context of how the thing is going to be used.
• 22:29
  And you need sector specific guidance, there's guidance that's tailored for each department.
• 22:34
  Broadly speaking, how do you think about what success means?
• 22:41
  Success for me is actually pretty simple, it's just are we building good stuff?
• 22:45
  It's honestly what it comes down to is are we actually building the easy AI in ways that's actually useful for the company?
• 22:54
  Because if we don't know for example, that means okay, we don't have a system of tracking, so we can't credibly say that.
• 23:02
  If you've got projects being built that are very cool, but you're just spewing misinformation or leading to loads of unplanned job role changes,
• 23:09
  where it's just frustrating loads of employees.
• 23:12
  That's not great as like a business ROI outcome either.
• 23:16
  So you need something that's this, we building good things that are reliable, accessible, inclusive, scalable?
• 23:23
  Do people actually want to be interacting with them and using them?
• 23:26
  And if we're doing that, that we're actually just delivering value because AI is very exciting, but it is just a technology at the end of the day.
• 23:35
  What are some of the new developments that you're looking at now terms of new technologies or the way that AI is evolving?
• 23:44
  think one of the most interesting things it's going to be on human AI interaction.
• 23:48
  think this is something I find particularly interesting.
• 23:51
  It's just because we went from even conversational agents now with the multimodal stuff where you could start to speak to them.
• 23:58
  Then the next thing is just, they can basically be around you, AI companion, actually seeing things like this door in front of you, right?
• 24:04
  That you can't see off camera.
• 24:05
  Those sorts of interactions are going to be interesting because I think they're going to fundamentally influence us as human beings.
• 24:12
  Like for example, I'm severely colorblind.
• 24:14
  So sometimes before even knowing what to wear, because I've embarrassed myself publicly before where I've worn mismatching colors, just to be sure I take a picture and use an AI
• 24:22
  to be like, do these colors match?
• 24:24
  And the next thing would be interesting is like, if you over rely on that for every part of your life, can, it's an enabler, but it can become a crunch.
• 24:32
  But if in the future you say, should I wear this?
• 24:35
  the colors match?
• 24:37
  Then the AI says, they do match, but you should buy from this brand.
• 24:41
  year for $20,000 and x what x amount of did you find out that company has a strategic partnership with that brand that you didn't know about that's what i find really
• 24:50
  interesting is that influence and persuasion that ai is going to have on us over the next few years
• 24:54
  Yeah, and that's a tough ethical challenge in a lot of ways.
• 24:57
  The example you gave seems like a more familiar kind of issue in terms of corporate influence and disclosure and so forth, but the broad problem, seems like, goes well beyond
• 25:06
  that.
• 25:07
  Yeah.
• 25:07
  I think it does because even when you're looking at the ethics of large scale deployments of, example, is it ethically acceptable to use generative AI as like a therapy bond?
• 25:17
  Now for most people, it could be if you're experiencing milder it might just be a good assistive tool.
• 25:24
  But if you are someone that needs genuine medical help and you're relying on this and it's not helping you, it might be making you worse.
• 25:32
  How are we going to balance that trade off the tools that are so accessible?
• 25:36
  for the entire population.
• 25:38
  I make sure everyone is getting the experience they need.
• 25:42
  Because if you're just printing like a blanket tool out there that's the same for everyone, regardless of their own individual circumstances, there is a risk here that some
• 25:51
  will benefit more than others, oh
• 25:55
  And what about the, you know, going back to this area of robot ethics and autonomous vehicles and devices, um what do you see today in terms of the challenges in that area and
• 26:07
  how far along we are in terms of coming up with good answers?
• 26:11
  I'd in terms of what's happening next, I find really fascinating is actually going from interacting with AIs to actually merging with them.
• 26:20
  So the ethical dilemmas over things like transhumanism, robotic prosthetics, brain-computer interfaces, which is creating a whole new area of ethics around neuroethics
• 26:33
  and AI, right?
• 26:34
  Things like, do we need the right to free will now?
• 26:36
  Because I don't want a brain implant that could be hacked, for example.
• 26:41
  or the right to mental privacy, then I don't want companies to be able to give me brain adverts, that type of thing, unless I have like a premium sponsorship model.
• 26:49
  And for anyone that's seen the latest season of Black Mirror, think that they'll know what I'm referring to.
• 26:54
  But these types of things I think are really important.
• 26:56
  And even we can learn a lot from science fiction because they're great digital ethics training, because it's just, do we not get here?
• 27:04
  It's the question we're asking ourselves, right?
• 27:07
  How do we just not get to this scenario?
• 27:11
  I used Black Mirror in my ethics class for a number of years, the class I've been teaching since 2016.
• 27:18
  And I would always have the students, you know, watch an episode and then relate it to what we talked about in the class.
• 27:23
  And one of the reasons I stopped was the distance between what was actually happening in the real world and what you'd see in a Black Mirror episode was compressed so much that it
• 27:33
  was not like crazy science fiction.
• 27:36
  It was, well, sure, have temple right now.
• 27:38
  It doesn't sound all that hard to think about.
• 27:40
  um But you're right, we need to look forward.
• 27:44
  It's interesting um when you talk about some of the transhumanism things, that I think to most people still feels like off in the future, but it sounds like now is the time that we
• 27:55
  need to be considering those issues.
• 27:58
  I mean, it's already happening.
• 28:00
  Like you already have human trials, have company selling robotic prosthetics.
• 28:03
  I mean, this could be amazing if we do well.
• 28:06
  But there are ethical questions over it.
• 28:08
  When do you stop augmenting yourself?
• 28:11
  Or at what point do you start becoming human?
• 28:13
  Right?
• 28:13
  It's like the ship of thesis parable.
• 28:15
  It's our what point do you basically automate yourself, augment yourself into a place where the loss of just human essence of dignity.
• 28:24
  Because we've not done this before as a species.
• 28:26
  So we are experimenting as we go.
• 28:28
  I if we get it right again, this could be amazing, but the risks even in terms of just inequality of like, if certain populations have access to these technologies first, which
• 28:38
  is better medical treatment, global inequality would just be exacerbated a lot more.
• 28:45
  Yeah, broadly speaking, it sounds like you're relatively optimistic that organizations and governments and civil society can address the risks of these new developments, not just in
• 28:59
  terms of robots and neural technology, but AI generally.
• 29:04
  I think this is first time I've ever been called optimistic.
• 29:08
  The previous guest of this chateau was to a pretty douse.
• 29:13
  It's more, we're looking at all the challenges.
• 29:16
  We're looking at what's hard to do.
• 29:20
  It's a mixture, but at least what I've heard you say seems more comfortable that there are responses and there are ways that smart people can get to them.
• 29:28
  I think there are ways of doing it.
• 29:29
  The one thing that gives me, it's frustrating about human nature that gives me encouragement is that I think as a species, we're very good at doing things when we
• 29:38
  absolutely have to.
• 29:40
  Like during that assignment, when the deadline is literally around the corner, then everyone's a genius and they're getting stuff done.
• 29:45
  It's just sometimes you just need to kick up the backside to say, just get it done, please.
• 29:49
  And can we just do it lot quicker?
• 29:50
  And I think the world really just doesn't need credible experts working together.
• 29:57
  Being very open and sharing what's working and what isn't.
• 30:00
  Even when it comes to things like spec sector specific guidelines and compliance.
• 30:04
  Like even which bias and fairness protocols even work effectively for AI and recruitment.
• 30:10
  What's the difference of deploying this in the US where there might be specific regulation around affirmative action.
• 30:15
  Versus in the UK with the Equality Act.
• 30:19
  As until you start being open and sharing these things is that you develop the best practices development guys.
• 30:27
  And is there that level of openness that you're seeing?
• 30:32
  I think you get some right?
• 30:34
  I think that you get some because I think within energy, most of these companies are actually very open about what they're doing because they have the safety culture where
• 30:41
  this is like a non-negotiable for a license to operate that is not seen as as much as a commercial advantage.
• 30:47
  Although like if you can do governance well, of course, like you can deploy things quicker, it's seen as everyone has to be doing this as an industry.
• 30:55
  And if you look at the rise of like AI safety institutes and with the Trump administration, as we pull back or certain things, but
• 31:02
  These were good attempts to release signals to say, do we attract the experts from companies to be working with civil society to just develop guardrails, right?
• 31:11
  That actually effective and apparently proven.
• 31:14
  And I think we just need more of that.
• 31:16
  You see more really qualified people with experience working together, balancing ideas off of each other and being allowed to explore and implement things, but finding out how can
• 31:25
  you just make them scalable.
• 31:28
  I don't know, you sound like an optimist to me, but that's a compliment.
• 31:32
  The thing is just...
• 31:33
  There's what the field does to you over time.
• 31:36
  This is honestly shocking.
• 31:37
  I need to tell my colleagues that.
• 31:38
  They told you I'm not as negative as you said.
• 31:41
  Ocon, thank you.
• 31:41
  It's been really great talking with you.
• 31:44
  This has been the road to accountable AI.
• 31:46
  If you like what you're hearing, please give us a good review and check out my Substack for more insights on AI accountability.
• 31:53
  Thank you for listening.
• 32:02
  This is Kevin Wehback.
• 32:03
  If you want to go deeper on AI governance, trust, and responsibility with me and other distinguished faculty of the world's top business school, sign up for the next cohort of
• 32:12
  Wharton's Strategies for Accountable AI Online Executive Education Program.
• 32:17
  Featuring live interaction with faculty, expert interviews, and custom-designed asynchronous content, join fellow business leaders to learn valuable skills you can put to
• 32:26
  work in your organization.
• 32:28
  Visit execed.warden.upenn.edu flash A-C-A-I for full details.
• 32:35
  I hope to see you there.