Latest / The Best of the Money Show / The Money Show Explainer: The AI accountability gap facing South Africa’s financial sector
Transcript
- 0:00Speaker 1: Across so many of our conversations on the Money Show,
- 0:03Speaker 1: you'll hear questions about how AI is changing finance. I mean,
- 0:07Speaker 1: last night you heard the Standard Bank CEO sim Schabalala
- 0:09Speaker 1: discussing our AI is changing his institution. Amid all of this,
- 0:13Speaker 1: it's clear that across all of our financial institutions, artificial
- 0:17Speaker 1: intelligence is becoming more involved in decisions, and decisions have consequences.
- 0:22Speaker 1: Who gets alone, who doesn't? Where's money put? Why is
- 0:24Speaker 1: it put there? You can imagine the potential for mistakes here.
- 0:28Speaker 1: Trading is moving more and more quickly. AI can move
- 0:30Speaker 1: money very very quickly, and any mistakes could multiply exponentially. Well,
- 0:35Speaker 1: someone who's been wrestling with all of this is in
- 0:37Speaker 1: fact the chairer of the financial is of the a
- 0:40Speaker 1: FFCA or Nati cumalana, and he joins us now or Nati.
- 0:43Speaker 1: Good evening, and thanks so much for your time. I
- 0:45Speaker 1: really do appreciate it. I mean, you've been looking at
- 0:48Speaker 1: this and it's quite complicated. I mean, if there's a
- 0:50Speaker 1: mistake or an event that creates huge problems, who's responsible?
- 0:54Speaker 1: Would it be the institution, the software person, the AI model?
- 0:58Speaker 1: How are you looking at the.
- 1:00Speaker 2: Problem so So the way that we are addressing and
- 1:04Speaker 2: approaching the problem is that there must always be a
- 1:11Speaker 2: human who is responsible either to deploy the technology or
- 1:18Speaker 2: to sign off on its deployment. There must be a
- 1:22Speaker 2: person who interacts with the regulator when is responsible for
- 1:28Speaker 2: therefore regulatory outcomes. That's that's what we want to emphasize
- 1:33Speaker 2: that you cannot have a situation where even if you
- 1:38Speaker 2: have outcomes which are not ideal, you can't you can't
- 1:44Speaker 2: blame the machine, you can't blame.
- 1:47Speaker 3: The technology that is deployed. There are people who made
- 1:52Speaker 3: the core and those people are other ones that we
- 1:56Speaker 3: will hold responsible for the outcomes. That's basically the message.
- 2:01Speaker 1: Is it very different from the situation now you might
- 2:04Speaker 1: have a trader acting on balf of one of the
- 2:07Speaker 1: big financial institutions. They make decisions. We've seen this happen
- 2:11Speaker 1: in the past where maybe that means the institution breaks
- 2:14Speaker 1: the law or something. Think of the currency manipulation case
- 2:16Speaker 1: that went on forever. Who is held responsible there? Is
- 2:19Speaker 1: it the institution, the CEO, the actual individual trader.
- 2:25Speaker 3: I think you know correctly, it should always be who
- 2:30Speaker 3: ultimately had the power to sign off on a transaction. Right,
- 2:37Speaker 3: so many institutions have got delegations of authority, and all
- 2:43Speaker 3: delegations of authority make it very clear that that I
- 2:47Speaker 3: have delegated a responsibility to you does not absorb me
- 2:53Speaker 3: from the fact that I am originally accountable and ponsible
- 3:00Speaker 3: for for what happens with this delegation. And I think
- 3:04Speaker 3: I think there's there's quite an important principle Steven that
- 3:09Speaker 3: senior executives of licensed institutions need to understand what it
- 3:15Speaker 3: means to be appointed formally as they are and to
- 3:23Speaker 3: pass fit in proper assessments UH of a regulator, and
- 3:29Speaker 3: then to be responsible on today to day. Whether you
- 3:33Speaker 3: have internal arrangements around who you delegate those functions too,
- 3:38Speaker 3: that's for convenience, that's for efficiency internally, but you are
- 3:43Speaker 3: ultimately responsible if things go wrong, if that if the
- 3:48Speaker 3: technology that's that's deployed, for example, has a bias in
- 3:54Speaker 3: terms of what the outputs are, you have got to
- 3:57Speaker 3: be able to explain how how the technology operates, and
- 4:02Speaker 3: you've got to be able to explain what the institution
- 4:05Speaker 3: will do to correct outcomes that are not in line
- 4:09Speaker 3: with the regulatory framework.
- 4:11Speaker 1: When things go wrong and someone's lost a fortune or
- 4:14Speaker 1: committed an illegal act. I can imagine in an institution
- 4:17Speaker 1: there can be a huge fight about who it was,
- 4:19Speaker 1: and someone can say you didn't issue the instruction, or
- 4:22Speaker 1: you did, or it was the software person who got
- 4:24Speaker 1: it wrong, or the code just made a mistake. It
- 4:26Speaker 1: wasn't foreseeable. Do you think institutions understand where responsibility ultimately lies.
- 4:34Speaker 2: I think many executions do.
- 4:36Speaker 3: I think that. Of course, at the point where there
- 4:40Speaker 3: is an issue, people will lawyer up and complicate the
- 4:46Speaker 3: picture in terms of who you hold accountable. But the
- 4:52Speaker 3: point that we are making in these kinds of communications
- 4:55Speaker 3: is to say, actually, it's not complex at all. Or
- 5:01Speaker 3: there are individuals who are responsible for outcomes of firms.
- 5:06Speaker 3: They are either executive directors or non executive directors, and
- 5:12Speaker 3: these established law or what their responsibilities are, and that's
- 5:18Speaker 3: what we will stick with. Regardless of what technology is deployed.
- 5:24Speaker 3: The people who are charged with governance outcomes of firms,
- 5:30Speaker 3: including firms that are licensed in the financial sector, continue
- 5:36Speaker 3: to be responsible for outcomes of those financial institutions, regardless
- 5:43Speaker 3: of whether what we're talking about is technology or other outcomes.
- 5:49Speaker 3: In the same way that as they publish results and
- 5:55Speaker 3: so on, performance of the organization, which includes areas as
- 6:00Speaker 3: where they would not necessarily have set foot themselves. But
- 6:05Speaker 3: because they're responsible for the overall, they're able to take
- 6:11Speaker 3: credit for what the performance is. If things go wrong
- 6:17Speaker 3: around regulatory expectations, the same.
- 6:21Speaker 2: People must be must be.
- 6:25Speaker 3: Responsible for the outcomes.
- 6:28Speaker 2: At this point, it seems to us it makes sense.
- 6:31Speaker 1: Yeah, I mean, at this point from what you can
- 6:33Speaker 1: see now, is there a need to update the way
- 6:36Speaker 1: you regulate markets and products. I mean, you could allow
- 6:40Speaker 1: precedent from previous decisions about how individuals must be responsibility
- 6:44Speaker 1: to just sort of follow, and I imagine that would work.
- 6:46Speaker 1: But do you think things need Do you think regulations
- 6:49Speaker 1: need updating to take account of AI's impact on financial services?
- 6:54Speaker 3: Not certainly. The point that that we made during the
- 6:58Speaker 3: lecture that gave recently at UCITY is that we have
- 7:03Speaker 3: to make sure that our regularly frameworks keep up with
- 7:08Speaker 3: the pace of change. But we also go a step further,
- 7:12Speaker 3: which is that we know that it's difficult for them
- 7:16Speaker 3: to keep pace. So therefore we must go into a space,
- 7:21Speaker 3: which is a space of principles rather than rules, because
- 7:27Speaker 3: principles tend to be relevant for much longer than than
- 7:32Speaker 3: rules would be relevant. And as of course, as we learn,
- 7:37Speaker 3: we updated regulatory framework, but our experience that is reactionary
- 7:45Speaker 3: responsive rather than keeping pace. So so principles are the
- 7:53Speaker 3: instrument that we think is better to have in the
- 7:58Speaker 3: formal regulatory frame work so that you're not left behind.
- 8:04Speaker 3: And principles talk to not just a specific technology or innovation.
- 8:11Speaker 3: They just talk to expectations in terms of what is
- 8:15Speaker 3: expected of a licensed entity, how they treat their customers,
- 8:19Speaker 3: what customers are entitled too, in terms of information, recourse mechanisms,
- 8:26Speaker 3: et cetera. And they're designed also around being proportional to
- 8:33Speaker 3: the size the nature of the complexity of the business
- 8:36Speaker 3: model of an institution. So that's how we think about
- 8:40Speaker 3: what one should do as far as the ability frameworks concerned.
- 8:44Speaker 1: Unatikam Lana, the chair of the Financial Sector Conduct Authority,
- 8:47Speaker 1: really appreciate your time tonight. Thank you. On the Many Show,