Latest / The Road to Accountable AI / Reggie Townsend: The Deliberate and Intentional Path to Trustworthy AI

Transcript

Reggie Townsend: The Deliberate and Intentional Path to Trustworthy AI · The Road to Accountable AI

• 0:02
  All right, Reggie, welcome.
• 0:03
  Thanks so much for joining us on the Road to Accountable AI.
• 0:06
  Thank you for having me.
• 0:09
  Tell me first what responsible AI or trustworthy AI, pick your favorite term.
• 0:14
  What does it mean to you?
• 0:15
  What does it mean to a company like SAS?
• 0:17
  Yeah, so I'm much more fan of trustworthy than I am responsible.
• 0:23
  I'm splitting hairs though.
• 0:26
  I think people get the sentiment.
• 0:28
  But I'll give you a quip.
• 0:32
  We don't say responsible electricity or responsible cars.
• 0:36
  We just say electricity.
• 0:37
  We say cars, right?
• 0:38
  And we expect electricity to be guarded sufficiently and to...
• 0:45
  conduct current rate as we might need it.
• 0:51
  We expect people to drive cars responsibly and we have standards and rules of the road, et cetera.
• 0:58
  And I think the same should be true with AI.
• 1:01
  Now, I do think the term trustworthy has a place because I think it's incumbent upon developers, providers of all sorts, people who deploy AI.
• 1:15
  to be worthy of trust, particularly in a time like now, where there's a fair degree of negative sentiment in some regards, and certainly a large degree of concern around the
• 1:33
  potential uses of AI.
• 1:35
  And so I think it's particularly incumbent upon us, like us here at SAS, to make sure that we are worthy of folks' trust.
• 1:42
  Now, we'll never.
• 1:43
  force anyone to trust us.
• 1:45
  You can't do that.
• 1:46
  It's not how trust works.
• 1:49
  But I do think that we have to prove ourselves worthy.
• 1:53
  And one of the ways that I think is necessary with respect to proving one's trust is to demonstrate that you are attempting to provide goods and services in this particular case,
• 2:06
  technologies, in ways that are going to advance
• 2:12
  other people, right, to help other people to thrive.
• 2:15
  And so that's why we really focus on this notion of human centricity, which we define as the focus on an individual's agency, equity and well -being, right, to ensure that each of
• 2:29
  us has the ability to thrive.
• 2:30
  There are a few other kind of principles that we revolve around, but that's the centerpiece, human centricity.
• 2:35
  Of course, we focus on things like inclusivity, privacy and security, robustness.
• 2:41
  and others, but that's how I see it.
• 2:45
  What does implementing those principles mean in practice?
• 2:48
  Because an analytics tool is an analytics tool.
• 2:51
  How is it going to be more or less human
• 2:54
  Yeah, so the principles to practice conversation is a big one in our industry.
• 3:02
  And the way we embody that is we went to a fair degree of granularity to define what we mean by certain principles.
• 3:13
  So I gave you a description of human centricity as an example.
• 3:16
  We have descriptions for all of our others as well.
• 3:20
  And so important in that is that
• 3:24
  Our principles might sound like some other principles that you'd find in other organizations, but culture shapes the language all the time, right?
• 3:31
  And so what we mean when we say transparency has to be different from what another organization might mean by transparency because of the nature of the kinds of businesses
• 3:43
  that we're in, right?
• 3:45
  That said, what we attempt to do in our products, in our business processes, and in our people
• 3:54
  is to have then an expectation for how those principles show up in each of those areas.
• 4:02
  So I'll give you an example.
• 4:03
  When we talk about accountability, we're really talking about being able to proactively identify and mitigate against adverse impacts.
• 4:12
  That's how we define accountability.
• 4:15
  Well, we recently released model cards with the idea of providing information about AI models.
• 4:23
  that puts an individual in the position to be able to proactively discern first what's going on with the model, but then proactively discern what the best next steps are that
• 4:35
  they should take on the basis of the information provided on the model car.
• 4:38
  So model car provides you information about accuracy of the model, fairness of the model for the particular impact in individuals that the model might be built for.
• 4:51
  A concept called model drift so that gives you a sense for whether the model is actually adhering to the expectations that were previously set established and so.
• 5:01
  know, with that information, you know a data scientist or a member of the board can say all right, this thing is performing as expected or not, and therefore, I must take a
• 5:13
  certain set of actions right I can report.
• 5:18
  I can do some auditing, can whatever.
• 5:21
  And so that's just a proactive means.
• 5:23
  So that's for us is bringing that principle of accountability to life.
• 5:31
  Model cards are something that a number of organizations are starting to use.
• 5:35
  How much more do you think those need to be standardized so that people can compare across and ensure that they have the right information?
• 5:43
  Yeah, so we compare model cars to nutrition labels on food, right?
• 5:49
  Nutrition labels on food are standardized to a fair degree, right?
• 5:54
  There are certain things that in the US, the FDA requires and in other jurisdictions, they require, know, basics, sugar intake, sodium.
• 6:04
  Now, you can certainly argue that there are other aspects of nutrition that could or should be on
• 6:11
  nutrition label and the same will be true for a model card.
• 6:16
  We've done, I think, a really good job at putting key aspects of a model for the different stakeholders.
• 6:24
  I guess the one distinction and where the analogy starts to fall apart, Kevin, is with a nutrition label, all of us are human and we're intaking and for the most part, vitamins
• 6:37
  and minerals and nutrients of all sorts have
• 6:40
  typical impact on humans.
• 6:43
  Whereas with AI models, models can be built for and interpreted by a variety of stakeholders for a variety of different reasons.
• 6:52
  The model itself is relatively amoral, but if you build a model that has applicability in healthcare, that is fundamentally different than using that same model in, I don't know,
• 7:06
  transportation.
• 7:09
  Also within organizations, you have a variety of different stakeholders, a model card that speaks to the needs of a CFO or CEO are different than the needs of say a data scientist
• 7:24
  or data analyst.
• 7:26
  And so what we tried to do is build a card that was sufficiently useful and sufficiently descriptive for range of stakeholders.
• 7:37
  think we did a pretty good job.
• 7:38
  Now, I'm sure there will be some who will argue on the far end of the technical capacities will say, hey, I need X, Y, Z.
• 7:48
  Okay, great.
• 7:48
  Well, we got the ability to go there, but maybe not on a model card.
• 7:52
  And then there may be others on another end of the spectrum.
• 7:55
  And so I think you get my point.
• 7:57
  We're trying to lock in on the vast majority of consumers of a model and how...
• 8:06
  how they may interpret the information they're associated.
• 8:12
  You've talked about a few different elements of a trustworthy or responsible AI or AI governance initiative.
• 8:19
  What was for you the biggest challenge in terms of rolling this out and getting buy -in
• 8:27
  Well, the biggest challenge to be quite honest with you is just getting the people.
• 8:34
  In our space, talent comes at a premium and just getting the people necessary to kind of get heads down.
• 8:41
  but beyond that, there was the design process to serve the needs of that range of stakeholders that I just described.
• 8:56
  In our technology, in our platform, there are a lot of pieces of information that we could have grabbed to display on a model card.
• 9:09
  But we had to make some very deliberate choices about what was useful and what wasn't.
• 9:14
  What was going to be more difficult to provide versus less.
• 9:23
  what would perhaps be considered proprietary for our customers, because you never want to put a customer in a position where they're unintentionally displaying proprietary
• 9:33
  information.
• 9:35
  going through that level of analysis was probably the more difficult of it all.
• 9:40
  But then the coding it all is the next step.
• 9:45
  I'm not the one doing the coding, so it's going to be easy for me to say that was the easy part.
• 9:50
  But I do know that the team
• 9:52
  spent a fair amount of time on kind of that design upfront so that we had a really robust way of reporting out that again that we think was and we thought was memorable or
• 10:04
  meaningful I should say and digestible all the
• 10:11
  SAS is a company that's been around for almost 50 years.
• 10:13
  It's an incredibly long time in tech and this area.
• 10:17
  Does that make it easier, you think, to implement these kinds of responsible AI initiatives than for a startup that's just getting off the
• 10:27
  I don't know that I would say easy or hard.
• 10:30
  I don't know that that's the way I would capture it.
• 10:35
  Certainly we have a legacy that we proud of.
• 10:40
  We've learned a lot over the course of 48 years.
• 10:45
  You learn over the course of time, assuming you maintain the institutional knowledge, you learn a lot about what works, what doesn't work.
• 10:55
  what a meaningful cycle is like, you know, we're in this hype cycle of AI versus, you know, say the hype cycle of crypto, right?
• 11:04
  Like there's certain cycles that you move on and some that you don't.
• 11:08
  So I think that kind of wisdom is very useful when it comes to creating things like model cars, because now we're able to apply some of the learnings over the years of, you know,
• 11:24
  how a CIO actually responds to bits of information that get provided as an example.
• 11:32
  But if we were a younger company, a startup, in that mode, you'd kind of throw things against the wall and you see what happens.
• 11:41
  And you have the ability to be agile and respond to the needs of the market in a much more rapid way than we might.
• 11:54
  We're an organization that is going to likely not be the first to market with a lot of these capabilities, largely because we have a significant customer base, right?
• 12:06
  We have people who rely on what we do, many of which are in human impacting industries that are highly regulated, like financial services, like healthcare, like government.
• 12:20
  And so we have to be a lot more
• 12:24
  deliberate and intentional when we make major adjustments.
• 12:29
  We have to certainly take into account what their regulatory needs are and try to anticipate future needs.
• 12:37
  And so that necessitates a level of analysis and design thinking as I described earlier that, you know, if it were a startup, we probably wouldn't spend the time doing some of
• 12:53
  those things.
• 12:55
  What are your conversations like with customers talking about trustworthy AI?
• 13:00
  Most of them right now are in the formulation stages of things like governance boards and oversight committees and the like.
• 13:10
  I think, at least for those that I speak with, they've all come to some level of recognition that they will be impacted from a regulatory perspective.
• 13:22
  Many of these organizations are global, and so they do business in places like the EU.
• 13:27
  And so now that we've got the AI Act, they have to respond to that in kind.
• 13:32
  Many are anticipating it, as well they should, that there's regulatory action coming out of jurisdictions literally around the world.
• 13:41
  But beyond the regulatory, one of the things that I advocate, and for the most part get a warm reception to this idea, is that it's just good business practice if you wanna
• 13:52
  demonstrate yourself as a responsible organization.
• 13:56
  Something like,
• 13:57
  think the number is 69 % of consumers are more willing to not just trust a brand, but then be prepared to buy from that brand at a premium if they perceive that brand to be ethical
• 14:16
  with its AI.
• 14:17
  And so it's not just about morality.
• 14:21
  It's about market relevance.
• 14:25
  appeal to that need in the marketplace through just good solid business practice, I think we'll all be the better for
• 14:37
  And how has the explosion of interest in generative AI impacted your efforts in this area?
• 14:45
  Well, again, I think from a market relevance perspective, it's, you know, us being one of the, hesitate to call us leaders.
• 14:56
  I'll let others define that for us.
• 15:00
  But, you know, we certainly have been expressing our voice out in the marketplace and have gotten a very warm reception to a lot of the work that we're doing.
• 15:09
  And so that's been very, it's been, it's had a boing effect.
• 15:14
  I guess, is the way I would describe it.
• 15:18
  It also importantly helps in a couple of other ways that I think tend to get overlooked.
• 15:25
  One is employee retention, right?
• 15:28
  Because employees also want to work for companies that are perceived as ethical in their approaches to AI.
• 15:37
  And so that piece is very important for us, particularly in this.
• 15:42
  in a talent scarce environment.
• 15:44
  But then the corollary to that is it helps with recruitment.
• 15:48
  So you know, when I put out a job post, I mean, I a I get hundreds of responses to, you different job openings.
• 15:58
  And so, you know, when I talk to some of my peers, that may not always be the case.
• 16:03
  And so
• 16:06
  I'm proud of that, to be quite honest with you.
• 16:08
  And I think it's largely a consequence of the work that we've been doing and the way that we've been displaying it.
• 16:15
  What do you think it takes for a company to be convincing that they're actually walking the walk on AI ethics?
• 16:22
  Because it's very easy to say, but what is it that you think has been effective in that way for
• 16:28
  So the message matters.
• 16:32
  The message has to be clear.
• 16:36
  It has to come across as authentic, as does the messenger, right?
• 16:45
  And so it's really important that if we say we're going to do something, that we actually prove it, right?
• 16:52
  And so integrity in the marketplace is huge.
• 16:56
  Look, people are smart.
• 16:58
  And, you know, if you say you care about how AI gets used and that you want to make sure that it doesn't hurt people.
• 17:11
  But then the next week you, you know, announce a product that may be inconsistent with your prior statements.
• 17:22
  That goes a long way toward creating distrust.
• 17:26
  And so we're very, again, very intentional, very deliberate about what we say and how we say it, who gets to say it, and then proving what we've said.
• 17:38
  And so we've had a decent track record at that.
• 17:42
  Of course, I'll record this podcast with you and then something weird will happen.
• 17:48
  But important in that is it's not a matter of if there's a misstep.
• 17:55
  corporations are large, there are thousands of people, of course there's gonna be some inconsistencies, but you've gotta be able to earn the right for people to hear you again.
• 18:09
  And again, that matters not just with the message, but the messengers.
• 18:15
  And so that authenticity that I go back to, people being able to relate and...
• 18:24
  have some believability when they talk in forms like this and from stages and in print and you name it.
• 18:33
  We always wanna come across as authentic and approachable and interested again in how others thrive as a consequence of our work.
• 18:46
  Our founder and CEO, Dr.
• 18:48
  Goodnights, says all the time, we are people here using our resources to help other people.
• 18:54
  That's the name of the game for us.
• 18:56
  And that permeates our culture.
• 18:58
  And again, it's something that I'm very proud of.
• 19:03
  Let me ask specifically about inclusivity and bias.
• 19:06
  That's an area where a number of companies have had serious missteps.
• 19:10
  But as you know, there's fundamentally bias that's baked into the data.
• 19:14
  so especially if we're talking about something like generative AI, it's not that you can just flip an off switch and make it go away.
• 19:20
  So what have you found is the best way to deal with that challenge?
• 19:25
  Yeah, I think the conversation about bias has to broaden.
• 19:31
  I think there is a negative connotation that comes when we use the term bias, particularly in our space.
• 19:37
  And one of the things that I try to do in this, and I'm happy we're in a podcast so I can expand a bit as opposed to on a newspaper article or something.
• 19:48
  So let me have permission to expand a bit.
• 19:52
  human beings use bias for the purpose of cognitive processing, right?
• 19:58
  We truncate in order to be more efficient with our brain energy, literally, right?
• 20:07
  That's the physiology of it all.
• 20:10
  So bias in and of itself is much more of a human function than it is a negative thing or a positive thing.
• 20:19
  Now, of course, there are
• 20:22
  biases that can have negative impacts.
• 20:25
  And that's largely the kinds of conversation or kinds of things that we're talking about in this conversation.
• 20:36
  To your point, data is always biased.
• 20:42
  It either is biased based on timing, right?
• 20:48
  So you might have recency bias
• 20:50
  or bias based on age, you might have geographic biases of some sort, right?
• 20:57
  And I've gotten data from only a small swath of folks in the Midwest versus West Coast, what have you.
• 21:06
  Your bias may be on the basis of who, type of people, right?
• 21:11
  Did they all look alike?
• 21:12
  Did they all sound alike?
• 21:13
  Did they all go to same schools and all sorts of things?
• 21:17
  I like to say that data,
• 21:19
  is really just our recorded existence.
• 21:21
  So everywhere we show up, right, we're bringing our bias along with us.
• 21:27
  And therefore, when the data is captured about us, the bias shows up in the data, as you said, right?
• 21:37
  So then you get into the model development process, be it generative AI or any sort of AI, and you've got to use data to train the model.
• 21:49
  And so the issue isn't whether bias exists.
• 21:52
  The issue is whether the bias is having a negative effect on the model.
• 21:58
  And if the users of the models are making decisions that are as a consequence biased negatively, right?
• 22:09
  So, you know, the hot button is race.
• 22:14
  And so if we use
• 22:17
  race as a piece of data, which some could argue that we should, some would argue that we shouldn't, but race is there.
• 22:24
  And whether you use race as a variable or not, there's certainly proxies that can be used to determine race.
• 22:32
  So we know we can determine race based on many sets of data about people.
• 22:40
  Now, if you want to use
• 22:44
  that data to make a decision as to who you should preemptively send law enforcement to go and chat with, that would be a negative implication of the use of that data.
• 22:59
  But if you want to use that same data set to decide who you should preemptively provide some assistive therapy to, because that therapy is proving itself to be more efficacious
• 23:14
  a particular race of people, then you should do that, right?
• 23:17
  That's for the benefit.
• 23:20
  And so I think we have to be really nuanced and honest about how we use data, why we use data, and for what purpose that data will be utilized.
• 23:36
  so look, these are grownup conversations, right?
• 23:39
  And if
• 23:42
  jump into them with this idea that all data is biased or all bias is bad, that's just the wrong starting point.
• 23:52
  We have to understand that what we're doing is human impacting and we have to make a decision.
• 24:00
  And if we want to, and this is why that definition becomes important, if we want to advance human agency, equity and wellbeing, right, to help people thrive,
• 24:11
  then our orientation, the use of that data and the use of the models that result from it are entirely different potentially than if we were simply using that data to pursue some
• 24:26
  profitable means.
• 24:29
  No, it's a really interesting way of putting it.
• 24:33
  In the world that we live in though today with so much sensitivity, is there hope that organizations can have those grown -up conversations?
• 24:42
  Always hope.
• 24:45
  There's always hope.
• 24:47
  You know, I think that literacy is hugely important in this space and not just technical literacy, but socio -technical literacy.
• 25:01
  You know, in our nation, we've been having a really difficult time with history lately.
• 25:07
  And, you know, my hope is that
• 25:12
  We will figure out a way through that.
• 25:19
  You know, catch me on the wrong day and I can be really cynical about it.
• 25:24
  But I maintain hope that we'll get to a point at which we can, you know, have smart, productive, and well -intended conversations about helping all people.
• 25:45
  whatever it is that we have to provide.
• 25:46
  In this case, we're obviously talking about AI.
• 25:51
  But it's a tough set of conversations.
• 25:56
  I will be very frank with you, Kevin.
• 25:58
  But there are conversations that need to be had.
• 26:02
  we've certainly had those conversations at SAS.
• 26:05
  We continue to have those conversations.
• 26:08
  And we've put...
• 26:13
  procedure in place to allow us the freedom to have those discussions in ways that we think are useful and beneficial and consistent with our values so that we can show up in the
• 26:24
  marketplace and compete on those values.
• 26:28
  Can't speak for everyone else.
• 26:33
  You serve on the National AI Advisory Committee for the White House.
• 26:37
  What are some of the things that that organization is working
• 26:41
  Yeah, so we are at this point in time focused on providing recommendations to the President and the White House on all matters of AI, where it intersects with society and
• 26:54
  innovation and economics and the way the government works and so on.
• 26:59
  We to date have offered about 18 or so specific recommendations, a collection of findings and objectives and
• 27:11
  what have you.
• 27:13
  The executive order from last October actually reflected some of the recommendations that we extended.
• 27:22
  And so right now, the focus is on a lot of gathering of information from the public so that we can offer a new set of recommendations.
• 27:36
  In fact, this week we'll be having a
• 27:39
  a public session related to workforce.
• 27:43
  Over the last few weeks, we've had similar sessions related to law enforcement, international cooperation, and a few others.
• 27:54
  And so what we do after those exercises are done is we do a lot of kind of data, know, fat gathering and consulting of experts
• 28:06
  At that point in time, we'll start to put our heads together as to what the incremental to the previous recommendations ought to be.
• 28:17
  that government will have the capacity that it needs
• 28:25
  No, but I think it's pretty well documented that the government itself has said that it doesn't have full capacity.
• 28:32
  Now, we should clarify what that means.
• 28:35
  You know, generally it's saying it doesn't have the expertise to appreciate some of the finer points of the technology.
• 28:45
  And so that's why it relies on what we might describe as a public -private partnership.
• 28:53
  That's why I rely on advisory committees like ours that are comprised of a number of folks from industry, but as well as civil society and academics and what have you.
• 29:07
  But our government is not unlike any other government around the world, right?
• 29:10
  I just came back from Australia talking to government officials, Japan talking to government officials, and they all say the same thing.
• 29:20
  And it kind of stands a reason, right?
• 29:22
  I mean, these people are in the business of government.
• 29:28
  know, bureaucrats, regulators, and I don't mean, you know, use those terms in any disparaging way, but that's what they do.
• 29:40
  They are not, you know, AI developers and scientists.
• 29:45
  so they have to rely on people who do those sorts of things.
• 29:49
  also don't have, we would probably argue lower our taxes if they did, they don't have the money to pay for a lot of those folks in mass.
• 30:02
  And so it only stands to reason that they have to go where the expertise exists so that they can adequately understand, so that they can then adequately put the necessary
• 30:13
  guardrails in place to make sure that the markets that they're,
• 30:18
  know, here to help to serve are operating in ways that are beneficial for the, in our case, the American public.
• 30:27
  Let me just ask you in closing, what gives you the greatest hope that AI development will proceed in a healthy
• 30:36
  Well, you know, I'm a bit of a fan of history and, you know, when we look at the history of technology development, Kevin.
• 30:47
  By and large, technology has been a net good for the humankind.
• 30:54
  Fluoride toothpaste, at one point in time, was an advanced technology.
• 31:03
  bathrooms, Inside bathrooms and refrigeration and vehicles and electricity and the internet and you go on and on.
• 31:13
  And in all of those cases, there was a moment in time where people doubted it, where people rejected and pushed against it.
• 31:25
  Of course, they were always the advocates who wanted to go faster, go further.
• 31:32
  But there's always a transition time.
• 31:35
  I mean, I can remember when the internet was coming on board and this thing called World Wide Web was taking everybody by storm.
• 31:45
  And you might remember this.
• 31:48
  Will we ever be able to transact online?
• 31:52
  Like, is it secure?
• 31:55
  Yeah, who would put their credit card number into a computer?
• 31:58
  What kind of idiot, right?
• 32:02
  And we look at now, you know, we wouldn't have Uber, you know, we certainly wouldn't have Amazon to the degree that it exists today.
• 32:12
  If if that weren't now true.
• 32:15
  And so I think AI, which, by the way, is a relatively old technology, as you know, it's been around for years.
• 32:23
  Of course, the generative.
• 32:25
  The way we use generative is somewhat novel, but the underlying tech is well proven.
• 32:33
  Neural network has been around for a long, long time.
• 32:38
  So I think this AI transition that we're in will pass, just like any other technical development has passed.
• 32:47
  What's critical in this moment in time, however, is that because of its ubiquitous nature,
• 32:55
  I believe we've got an opportunity to reimagine structures.
• 33:02
  And so if decisions that we've made in the past or a function of whatever our thinking was in the past that eventualize in the structures of our past and present, we're at a moment
• 33:19
  in time where we can say, all right, we're going to start automating decisions through this thing called AI.
• 33:25
  And we're going to eventually create some new structures.
• 33:29
  And so do we want those new structures to be greatly beneficial for all or be greatly beneficial for a few?
• 33:41
  And I think the consternation that we feel and the concern that we see and feel out in the marketplace, out in the world really, is really getting at that fundamental question.
• 33:54
  Is this being built for me with my needs in mind or not?
• 34:03
  And so I think we have an opportunity in this moment in time to say, yes, it is being built with your needs in mind.
• 34:12
  Yes, it will help you stay secure.
• 34:15
  Yes, it will help you thrive into the future.
• 34:18
  But we've got to be able to say that in an authentic way.
• 34:24
  We've got to mean it, and then we've got to prove it.
• 34:27
  And that's all of us, those of us who are developing, deploying, governing, using, you name it.
• 34:36
  And so we have a unique moment in time, and this moment will pass.
• 34:41
  So I just hope that we take the moment and we use it to the greatest good.
• 34:48
  Reggie, thanks so much for taking this moment to have this conversation with
• 34:52
  I do appreciate you.
• 34:53
  Thank