Latest / The Road to Accountable AI / Paula Goldman: Putting Humans at the Helm

Transcript

Paula Goldman: Putting Humans at the Helm · The Road to Accountable AI

• 0:00
  Paula, welcome to the podcast.
• 0:02
  Really a pleasure having you here.
• 0:03
  Thanks for having me.
• 0:05
  You were one of the first people, I think, who was hired in a role like yours,
• 0:11
  dealing with ethical and humane use of technology in a major company.
• 0:16
  How did you first get started?
• 0:18
  What were the first things that you did in setting up this program at Salesforce?
• 0:23
  Yeah, I came on board at the beginning of 2019.
• 0:29
  And if we can sort of rewind back to that moment in time, I think there were a lot
• 0:36
  of swirling questions about the role of technology.
• 0:40
  They were not necessarily all specific to AI per se, or at least that wasn't the
• 0:45
  language that was being used at the time.
• 0:48
  But I think to Salesforce's
• 0:51
  credit, they foresaw that these weren't sort of one -off questions, but more that
• 0:57
  this was a practice and a discipline that needed to have leadership across the
• 1:03
  company.
• 1:03
  And so they created this Office of Ethical and Human Use at Salesforce.
• 1:08
  But the role was, it wasn't necessarily super well -defined.
• 1:14
  And so what we did at first was a couple of things.
• 1:18
  We worked on
• 1:20
  really clarifying principles and initial set of policies, customer -facing policies
• 1:26
  around product and use.
• 1:28
  But equally, we really focused internally on culture and employee participation,
• 1:35
  knowing that there's no sort of team or set of individuals that has all the
• 1:41
  answers or can anticipate all of the uses of technology, but really starting with
• 1:47
  building a culture where
• 1:48
  this is everyone's responsibility and everyone should be asking these questions
• 1:52
  was a really strong starting point for us.
• 1:57
  How do you build that kind of culture?
• 2:00
  Well, you find channels for employees to raise questions, both either, I'm going to
• 2:08
  answer that again, I'm going to start again.
• 2:12
  Well, it can start with finding lots of different channels for employees to ask
• 2:16
  questions, right?
• 2:18
  And some of those might be formal channels and some might be informal, some might be.
• 2:24
  with their names attached and some might be confidential.
• 2:26
  So for example, we use a sort of ethics point reporting channel where employees
• 2:32
  can confidentially ask questions or report concerns about the uses of our technology.
• 2:39
  We have a sort of process for vetting those types of things.
• 2:43
  We invite employees to sit on our Ethical Use Advisory Council, which also has
• 2:48
  executives from across the company and external experts.
• 2:52
  And then it's really about sort of the tone from the top, holding participatory
• 2:58
  events and finding ways to sort of bubble up these themes across the org.
• 3:05
  You know, and it's an ongoing effort.
• 3:07
  It's never really done.
• 3:09
  You mentioned that your role is broader than AI.
• 3:13
  How specifically does AI fit in with this framework of ethical and humane use?
• 3:19
  And what's distinctive about the kinds of practices you're doing around AI
• 3:24
  responsibility?
• 3:25
  Yeah, well, I will say AI is a huge, huge, huge focus and particularly in the last
• 3:31
  few years, but we've been working on responsible AI for, I don't know, six or
• 3:34
  seven years at this point.
• 3:36
  And.
• 3:39
  And the only reason that I said that the role is broader is because I think we
• 3:43
  sometimes forget when we talk about AI, AI is also about data.
• 3:48
  It's also about sort of the interfaces that are built on top of these models and
• 3:54
  whatnot.
• 3:54
  So I'm responsible for making sure that everything, that all the products that we
• 4:01
  ship out into the world are good for the world, so to speak, for...
• 4:06
  mitigating any unintended consequences or negative consequences and finding all the
• 4:13
  positive use cases and amplifying them.
• 4:16
  But I will say AI is really core to that.
• 4:21
  And particularly when we think about heading towards AI agents and whatnot and
• 4:27
  like very complex set of questions that come with that.
• 4:31
  Yeah, so what are some of the ways that your program has evolved as AIs come to
• 4:37
  the forum or we now have generative AI being significant piece?
• 4:41
  What are some of the things that have changed along the way?
• 4:46
  It's interesting, you know, I think it was five or six years ago that we had a set of
• 4:53
  principles and an AI charter.
• 4:57
  There was a leader on the team named Kathy Baxter that led that process.
• 5:02
  And then a year ago, we looked up, or I would say maybe a year and a half ago, we
• 5:08
  looked up.
• 5:09
  as generative AI was becoming more and more in focus.
• 5:13
  And AI was not just making predictions, but creating content.
• 5:17
  And as we look forward, it's increasingly taking action on our behalf.
• 5:21
  And that brings with it not necessarily new risks, but a different set of risks.
• 5:28
  So when I say new, I mean more.
• 5:32
  I'm going to start that again, too.
• 5:37
  So when we look at generative AI and the transformation from AI making predictions
• 5:44
  to creating content to now more and more taking action on our behalf, the risks
• 5:50
  that come up are not necessarily new, but almost with a different accent, right?
• 5:54
  Different emphasis.
• 5:56
  And so, for example, with predictive AI, accuracy was an issue, was a question.
• 6:05
  But we weren't thinking about so -called hallucinations in the same way.
• 6:12
  And so right now, when we talk to customers about generative AI, that
• 6:17
  question, how do I know the answer is accurate, is at the top of the list.
• 6:21
  Similarly, when we think about toxicity, potential for generative AI to create
• 6:29
  answers that might offend, that also was not an issue so much.
• 6:35
  with predictive AI.
• 6:36
  So we had to create a new set of principles.
• 6:39
  And then working with our product team, so we have this whole suite of generative AI
• 6:43
  products across marketing, commerce, and so on, service.
• 6:49
  We work to create a set of guardrails directly in the product itself that are
• 6:57
  addressing these key questions around trust.
• 7:00
  Everything from,
• 7:02
  zero data retention, so making sure that our customer's data stays within the
• 7:08
  guardrails and does not go get used to train some third party model, to a
• 7:14
  toxicity detector that helps customers understand and set guardrails around that
• 7:21
  type of content.
• 7:24
  And there's so much more there.
• 7:25
  But it was really exciting for me and continues to be very exciting to work on
• 7:29
  those because these...
• 7:31
  show me these products show me that trustworthy AI as a goal is a set of
• 7:39
  innovations and it can really help to make product outcomes much better.
• 7:47
  So how much can these kinds of problems like hallucination, toxicity and so forth
• 7:53
  be solved by these technical mechanisms and how much has to go to the kinds of
• 7:58
  things you were talking about before, culture or other kinds of techniques?
• 8:04
  It's a great question, and I think it comes down to context and specifics,
• 8:10
  right?
• 8:10
  So we, as much as possible, because we're an enterprise company, not a consumer
• 8:14
  company, we try to turn the controls over on these types of questions to customers.
• 8:20
  Well, take, for example, the question of toxicity.
• 8:25
  You just referred to it, right?
• 8:27
  So when you think about what type of content would you want or not want,
• 8:34
  you might have a brand that's super edgy.
• 8:37
  And so maybe a curse word here and there is okay.
• 8:40
  And then maybe you are a huge multinational consumer brand and that is
• 8:45
  not okay, right?
• 8:47
  We want to be as much as possible turning over those thresholds for customers to
• 8:52
  define for themselves.
• 8:53
  And that's where I think there's a sort of way of thinking about it, of defining the
• 8:58
  problem and of measuring it.
• 9:00
  But then you wanna be able to sort of let...
• 9:02
  that decisions be context specific.
• 9:05
  I would say somewhat similar answer to the question around accuracy, although as you
• 9:14
  know, I mean, I think there have been really rapid improvements with that as
• 9:20
  LLMs have gotten better and better and foundation models have gotten more and
• 9:24
  more advanced.
• 9:25
  And that's been a very rapid advancement.
• 9:28
  There are certain techniques and technical innovations, so to speak, that
• 9:32
  has been incredibly helpful across our customer base.
• 9:36
  One of them is retrieval augmented generation, essentially this notion of
• 9:40
  saying instructing the model not to answer a question unless it finds the answer in
• 9:45
  some trusted data.
• 9:48
  But then also, there's also room for context, right?
• 9:53
  So you might want a really creative answer to your question, right?
• 9:58
  And so you might want, for example, to set the temperature on the model that you're
• 10:04
  using to be very, very flexible and creative, or you might want only a very
• 10:08
  precise answer to the question.
• 10:09
  So I still think it's important to keep sufficiently aware of the answers to these
• 10:15
  questions being context.
• 10:17
  Yeah, I know it's really interesting.
• 10:18
  Do you have to do a lot to educate many of your customers about those issues so they
• 10:24
  know what knobs to turn?
• 10:26
  Yeah, I mean, we work very, very closely with our customers on these topics.
• 10:30
  We provide a lot of in -product documentation and a lot of sort of
• 10:38
  generally available.
• 10:40
  Like we have a platform called Trailhead, which does free courses around these types
• 10:45
  of topics.
• 10:46
  And I would say it's not just our customers, right?
• 10:48
  It's the whole ecosystem.
• 10:51
  And in particular, I would say lots of folks, individuals just thinking about
• 10:56
  their own careers that are really hungry to think about these issues and how does
• 11:00
  it affect us at work and what's the future of generative AI at work.
• 11:05
  And I think these questions could not be more important at this moment in time.
• 11:10
  Let me ask you about a few organizational questions.
• 11:12
  One of the issues that comes up a lot is where do you put a group like yours?
• 11:17
  Does it go in legal or compliance?
• 11:20
  Does it go in product?
• 11:21
  Does it go in lots of other places?
• 11:23
  I know you went through an evolution about where you sit within Salesforce.
• 11:27
  I'd love to hear about how you think.
• 11:29
  Yeah, as it turns out, I actually, I sit in both orgs at the same time.
• 11:35
  I sit in product and in legal.
• 11:37
  And that's extremely useful because in fact, there's so much overlap.
• 11:43
  Well, that is to say, when you have a group that's really looking at the impact
• 11:47
  of product in the world, you want to be there in the product development life
• 11:51
  cycle.
• 11:51
  Hence sitting in product, being really deep in the life cycle of product
• 11:56
  development and whatnot.
• 11:58
  And at the same time, it is legal and government affairs.
• 12:02
  There's so much regulatory activity happening right now.
• 12:05
  There's so many really important and encouraging conversations happening in the
• 12:09
  public.
• 12:10
  And the intersection of that with legal policies and stuff like that needs to also
• 12:15
  be really at the forefront.
• 12:16
  So it's a pleasure to be able to go and operate across both works.
• 12:21
  And really, honestly, we operate across the whole company because we have to and
• 12:26
  we need to.
• 12:28
  How do you build those relationships and connections, especially with the product
• 12:32
  groups, they want to ship something, they want to get out there, they want to meet
• 12:34
  customer needs, and you may be slowing them down, asking those kinds of
• 12:39
  questions.
• 12:41
  So I see a lot of the work that we do is around translation.
• 12:48
  So there's a lot of foundational work that's like, how do you assess these
• 12:51
  risks?
• 12:52
  What are the technical mitigations for the risk or the policy mitigations for the
• 12:56
  risks?
• 12:56
  How do we think about all of that as a suite of tools together?
• 13:02
  And then a lot of it is translating to.
• 13:05
  OK, what's the goal for this product?
• 13:07
  What's the business goal for the product?
• 13:08
  What are the customers trying to achieve?
• 13:10
  Why do these mitigations, these innovations make these outcomes better?
• 13:15
  And then how do you think about even stacking and ranking some of these goals
• 13:23
  as they come together?
• 13:25
  And so we spent a lot of time on that and posing these questions to cross
• 13:31
  -functional groups of
• 13:34
  of stakeholders internally.
• 13:36
  And a lot of it is relationship building.
• 13:39
  But going back to that word translation, it's about taking these questions and
• 13:44
  putting them into the framework that these groups are already operating on and
• 13:49
  helping them understand why these are such important questions to be asking.
• 13:54
  And I will say that.
• 13:57
  has become super clear in the last couple of years because generative AI, you know,
• 14:06
  I think one of the interesting things that's happened is that as this technology
• 14:13
  captured public imagination and people were playing around with publicly
• 14:18
  available versions of the technology,
• 14:21
  It was at least the first time in my memory that we're talking about the
• 14:25
  promise and potential of the technology at the same time as we're talking about in
• 14:30
  the very early days, the risks of the technology and those risks are top of mind
• 14:34
  for all of our customers and for individuals.
• 14:37
  And so it makes that conversation internally very fluid and very easy to
• 14:43
  have because it's not coming out of the blue.
• 14:48
  What are the risks that are most compelling for those stakeholders that
• 14:53
  really encourage them to see the need for these kinds of interventions?
• 14:57
  You mean internal or external?
• 15:00
  I'm curious if it's a different answer for each of them.
• 15:03
  Well, I will say our customers, and again, remember we serve mostly businesses large
• 15:08
  and small.
• 15:10
  I would say the question of the integrity of their data is top of mind, making sure
• 15:16
  that their customer confidential data stays confidential and that they're in
• 15:22
  control of it.
• 15:23
  That is followed by the fact that we have a lot of data that is not
• 15:26
  probably by, I mean, maybe that goes hand in hand, I should say, with the questions
• 15:30
  that we were talking about earlier around accuracy and being able to then sort of
• 15:36
  subsequently control sort of thinking about tone and toxicity and those types of
• 15:42
  questions as they come up.
• 15:46
  Salesforce is in large part, again, we're helping customers connect with their own
• 15:52
  customers.
• 15:53
  We're helping companies connect with customers, right?
• 15:55
  In large part, we're not talking about high risk decision making for the most
• 16:01
  part.
• 16:01
  So we're not talking about deciding whether someone gets a job or a loan or
• 16:07
  access to housing, that kind of thing.
• 16:10
  And so it's not quite the same as you might, those discussion of risks as you
• 16:16
  might find with some of those higher risk use cases as exist in the world of AI.
• 16:21
  What else is distinctive about Salesforce's culture that's relevant to
• 16:27
  the work that you do?
• 16:29
  Well, I'd say it's been a mission -driven company from the very beginning.
• 16:33
  From the very beginning, trust has been our number one value.
• 16:35
  It's never moved.
• 16:39
  And Salesforce, I think, had to walk that talk really early because, if you may
• 16:46
  recall, it was introducing cloud technology at a time when people were
• 16:52
  skeptical of it, right?
• 16:54
  And so it had to both...
• 16:58
  It had to walk the talk in the sense of product innovations, but also in terms of
• 17:03
  consistently following a set of values.
• 17:05
  And an example of that is very early on the company put up a trust site.
• 17:13
  So one of the objections to putting customer data or putting a company's data
• 17:18
  into the cloud was, well, what if it.
• 17:21
  fails?
• 17:21
  Like, what if it's not available?
• 17:23
  Right?
• 17:24
  And so very early on, Salesforce put up this site that very transparently put up,
• 17:30
  you know, statistics about outages and when it was up and when it was down.
• 17:35
  It's that kind of thinking that has been with the company for so long.
• 17:42
  And I see this as an extension of it.
• 17:45
  You know, there's not necessarily, as you know very well, Kevin, there's not
• 17:48
  necessarily yet.
• 17:50
  a set of commonly recognized standards for how to do at least the generative part
• 17:58
  well.
• 17:58
  I think it's evolving quickly.
• 18:01
  But we're just sort of pushing the frontier on it within the company and
• 18:05
  working with stakeholders outside the company to keep learning and sharing best
• 18:09
  practices back and forth.
• 18:12
  How do you think about what trust means in this context of responsible AI?
• 18:16
  It's a great example in the early days of the cloud where trust is just, is my data
• 18:19
  there or not?
• 18:20
  Is the thing up or not?
• 18:22
  Everyone loves to talk about trust and I've written books about trust and I think
• 18:25
  it's critically important in the AI context, but how do you understand exactly
• 18:28
  what that means?
• 18:29
  Well, I think it's the questions that we were just talking about.
• 18:34
  Is this outcome accurate?
• 18:38
  Does it align with my brand?
• 18:44
  Is it giving the right information to the right customer at the right time?
• 18:48
  These are very standard things that you might expect from the type of technology
• 18:54
  that we develop.
• 18:55
  Now, there's, I think, a meta.
• 18:57
  There's two layers I might add onto that.
• 18:59
  I'm curious how you would define that.
• 19:01
  You would answer this question.
• 19:04
  But now I'm going to blank on the name of this woman where you're going to have to
• 19:09
  come back to this and re -record this part of it.
• 19:12
  You might be able to place this quote of trust being a confident relationship with
• 19:16
  the unknown.
• 19:18
  Do you want to?
• 19:19
  Rachel Botsman.
• 19:21
  Yeah.
• 19:21
  I would add a couple of layers onto what I just said.
• 19:24
  So one I would say, I think that there's a sort of meta layer to trust for AI right
• 19:30
  now.
• 19:32
  I believe it was Rachel Botsman who said that defined trust as a confident
• 19:36
  relationship with the unknown.
• 19:38
  I think our customers understand that this technology is moving very quickly and
• 19:43
  that.
• 19:43
  the questions that they're asking today may be different than the questions that
• 19:47
  they need to ask tomorrow.
• 19:49
  And some of this is just about the relationship that we build overall over
• 19:53
  time that do we continue as a company to keep asking the right questions, being
• 19:58
  transparent about what we're learning, where we got it right and where we got it
• 20:01
  wrong.
• 20:03
  So I think that is very much the kind of meta part of what's going on with AI right
• 20:08
  now.
• 20:08
  And I would also say, and maybe this is kind of follows directly from that,
• 20:13
  we're really at an inflection point as we think about AI agents because that is a
• 20:19
  very, very different set of outcomes.
• 20:23
  And it is a very different way of interacting with AI as a potential to just
• 20:27
  really upend how we interact with technology overall.
• 20:33
  And so for that, when we think about a single prompt, for example, generating an
• 20:40
  entire email campaign that might
• 20:43
  generate a million emails, we're thinking very much about, OK, so how do we evolve
• 20:49
  the set of controls that we give customers to be able to trust that?
• 20:54
  And some of it is about the new control panels.
• 20:59
  So the control panels that might, if you think about the history of technology and
• 21:03
  the design of, for example, control panels for airplanes so that pilots didn't make
• 21:07
  errors, that kind of thing.
• 21:09
  I think it's a similar moment for.
• 21:11
  AI agents right now.
• 21:13
  And what we've been working on is these sort of pre -control panels where you
• 21:19
  might be able to say, here's my prompt and get a preview of what it looks like and
• 21:23
  turn the dials up and down on tone and whatnot.
• 21:26
  And then post -control panels, we have an audit trail in our trust layer.
• 21:32
  And that audit trail will sort of show you what's been all the feedback on the
• 21:36
  content generated and.
• 21:38
  You might be able to then identify anomalies and whatnot, and then go back
• 21:42
  and correct and keep fine tuning those outcomes.
• 21:45
  And I think very much directionally, that's where we have to be headed for AI
• 21:49
  is these sort of macro views of how it's going in addition to the micro, is this
• 21:54
  individual piece of content?
• 21:57
  Did it do what I wanted it to do kind of thing?
• 22:00
  You've been talking recently about the importance of humans at the helm instead
• 22:04
  of the, the usual formulation, humans in the loop.
• 22:07
  So, what, what exactly does that mean in this context?
• 22:10
  Okay.
• 22:12
  So for a long time with AI, we've been talking about this notion of human in the
• 22:17
  loop.
• 22:18
  And with generative AI, what that has colloquially translated to is I'm going to
• 22:25
  let a generative AI model draft a
• 22:28
  a note to, I'm going to send a note to Kevin, draft an email to Kevin that tells
• 22:34
  him what a great time I had on this podcast, right?
• 22:37
  And then I'm going to edit it before I send it over and just make sure it really
• 22:42
  represents me.
• 22:43
  Well, that may still be appropriate in a lot of cases, but as we move to the world
• 22:48
  of agents where AI is taking action on our behalf, it may not be practical in all.
• 22:55
  instances because they're getting better and better at executing action at scale.
• 23:02
  And so we need sort of the next set of controls, the next set of making sure that
• 23:09
  we like to say, you don't always necessarily need to have a person rowing
• 23:14
  the boat, but you absolutely need some person in charge of steering the ship.
• 23:18
  They need to know what's going on.
• 23:20
  They need to be able to verify it's going well, and they need some controls in case
• 23:22
  it's not going well.
• 23:24
  And that's what we call human at the helm, which is we like to see as a more powerful
• 23:29
  approach to keeping people in control of AIS as its powers continue to evolve.
• 23:37
  Yeah, the talk about agents and these concerns as the technology evolves, for
• 23:42
  me, brings up this whole AI safety debate and people who are focused on the frontier
• 23:51
  models and the even potential existential threats of AI.
• 23:55
  Do you worry that that discussion gets too much airtime and distracts from the kind
• 24:01
  of immediate practical issues with the technologies that companies like yours are
• 24:05
  actually deploying today?
• 24:07
  I guess I would say a couple of things.
• 24:09
  One is I think there's a little bit of a false dichotomy.
• 24:13
  And I wrote a piece about this in Foreign Affairs.
• 24:16
  But I spent a lot of time thinking about it.
• 24:19
  And it seems to me, or I feel very strongly, that the controls we develop
• 24:25
  today for the problems we're experiencing right now and the problems we anticipate
• 24:29
  coming down the road are how we build muscle memory for...
• 24:33
  and honestly technological insight into more and more complex forms of AI.
• 24:39
  And that's how it's always been.
• 24:41
  It's not like we're starting from scratch, for example, on trustworthy AI.
• 24:45
  We're building on years of study and years of innovation around topics like fairness
• 24:50
  and bias and so on.
• 24:52
  And that will remain true.
• 24:56
  So yes, I think that we're...
• 24:59
  to some degree overlooking the importance of these current day investments as they
• 25:05
  create the bridge to make sure that we, the trust innovation keeps pace with
• 25:10
  general innovation.
• 25:11
  So that's how I see it.
• 25:16
  And what about the torrent of regulation that's here now or coming around the world
• 25:22
  in this area?
• 25:22
  How does that impact on the work that you do?
• 25:25
  Well, I am honestly delighted to see this level of attention from regulators.
• 25:33
  Particularly happy about the framing of the EU AI Act.
• 25:39
  And I know there's still some stuff that is being worked out.
• 25:42
  But generally speaking, taking a risk -based approach and regulating AI based on
• 25:51
  particular types of outcomes or decision -making versus, you know, particular
• 25:58
  models, which I don't think is the, not to say I don't wanna cast it in sort of
• 26:03
  totally dichotomous terms because there's a place for focusing on very powerful
• 26:08
  models, particularly in kind of like national security.
• 26:12
  concerns and elements.
• 26:14
  But I think sitting where I do, where I'm serving the enterprise, there is a very
• 26:20
  big difference between the risk level of sending, drafting an email for a customer
• 26:24
  versus deciding whether that customer qualifies for a loan.
• 26:28
  And so I think that type of regulation is quite appropriate and helpful.
• 26:33
  And generally speaking, I sit on the National AI Advisory Committee, which
• 26:38
  advises the White House on AI.
• 26:39
  But I think there's been a number of steps forward as well in the executive order in
• 26:44
  the United States and whatnot.
• 26:46
  There's much more to go.
• 26:48
  Obviously, the political environment can be quite challenging.
• 26:52
  But it's really important that we have a set of guardrails.
• 26:56
  And I think that set of guardrails can unlock innovation, unlock companies going
• 26:59
  faster.
• 27:01
  Yeah, where do you think we're going in the US given that challenge, as you
• 27:05
  mentioned, of getting legislation through?
• 27:08
  Well, my hope, I will say, I think you gotta not forget the basics, right?
• 27:13
  We need a national privacy bill.
• 27:15
  We've gotten close a few times.
• 27:19
  Right, we need a national privacy law.
• 27:23
  We've gotten close and I'm cautiously optimistic about that.
• 27:32
  And then I think there's a number of, as you know, ongoing debates about treating
• 27:39
  AI sort of as its own regulatory concern versus as a kind of topic -specific
• 27:45
  regulatory concern, be it in medicine or in finance or whatnot.
• 27:51
  And I think, again, there, I think we need both as well.
• 27:55
  So.
• 27:56
  You know, we'll see a lot really rides on kind of what happens across the next set
• 28:02
  of elections and whatnot.
• 28:03
  But generally speaking, I've also been encouraged by the fact that it's a
• 28:08
  bipartisan area of interest.
• 28:11
  So you don't always get that.
• 28:13
  And I think that's great.
• 28:16
  And just in general, what do you see as the biggest challenges coming in this area
• 28:21
  of trustworthy or responsible AI in the next few years?
• 28:25
  You know, if you'll allow me to geek out for a moment, you know, certainly there's
• 28:32
  no shortage of questions and challenges, right?
• 28:35
  Be it disinformation that people are worried about, or be it just generally
• 28:41
  being able to trust the outcome that you're working towards and a lot of
• 28:48
  progress made on that latter piece.
• 28:51
  What I really think I'm thinking a lot about is the use of AI in the workplace
• 28:55
  and how important it is for the products we design and the ways that we integrate
• 29:02
  AI into the workplace to give people their own sense of what AI is good at and what
• 29:10
  it's not good at, and also what people are good at and where they may have their own
• 29:16
  cognitive biases that AI can really help with.
• 29:18
  So if we do this right,
• 29:20
  If we do this right, we can get, you know, we design for the strengths of both and we
• 29:27
  get better outcomes, right?
• 29:28
  We get, you know, doctors who get, you know, who are able to not have to spend
• 29:36
  time taking notes, but can check for inaccuracies and then also watch for
• 29:43
  subtle cues in the patient interaction that might not, you know, be evident.
• 29:50
  Otherwise, right?
• 29:51
  Or we get salespeople and service people, same thing, being able to really focus on
• 29:59
  the relationships with their customers and not have to spend the time looking up the
• 30:05
  context of what might, you know, a customer that might need to return an
• 30:10
  item.
• 30:10
  And they've got that all, all that information at their fingertips via an AI
• 30:14
  and in their CRM system.
• 30:16
  And so much more as we think about.
• 30:18
  agents entering the workplace.
• 30:21
  I think it's really important though that people really understand what they can
• 30:28
  trust and what they can't trust and they've got the appropriate tools for
• 30:31
  that.
• 30:32
  And it's not as often that we talk about that.
• 30:35
  It's not as often that we talk about also the sort of behavioral science piece of it
• 30:40
  and which interventions really help add what we call at Salesforce a little bit of
• 30:45
  mindful friction to the interface.
• 30:48
  and which of those really works that people know when they need to take a pause
• 30:52
  and take a second look and not just simply trust the outcome.
• 30:58
  Yeah, no, it's a really critical point.
• 30:59
  It's one of the things we have people at Wharton and other academic institutions
• 31:04
  doing really interesting research about that question.
• 31:06
  And I've been fascinated by even some of the work on fairness where we can look at
• 31:12
  the bias in the AI.
• 31:13
  But then in some cases, the human will reinforce the bias of the AIs.
• 31:18
  They'll disregard the AI when it's better and use the AI when it's not and so forth.
• 31:22
  So it sounds like though you're optimistic that we can build these kinds of systems
• 31:26
  that address those.
• 31:27
  I am, but we have to focus on it.
• 31:29
  Right, and what you just like, there's a lot of those types of examples that are
• 31:33
  counterintuitive.
• 31:34
  It's not just, there's lots of different outcomes we could get.
• 31:38
  It's not just, for example, that, you know, it's possible that we need to do
• 31:44
  more, not possible, we definitely maybe need to be doing more around thinking
• 31:48
  about correcting bias in AI.
• 31:50
  However,
• 31:51
  we should be thinking about designing these systems, given that human systems
• 31:55
  also have bias, right?
• 31:57
  And that you get these counterintuitive outcomes sometimes where people don't know
• 32:03
  how to spot that bias in the AI.
• 32:05
  So we really have to be thinking about this complex set of interactions.
• 32:11
  And yes, I'm really optimistic that if we're really paying attention to those
• 32:17
  dynamics and bringing in...
• 32:19
  I mean, there's a lot of literature on that topic that we can bring into how we
• 32:23
  design these products, but we need to accelerate that.
• 32:27
  We really need to pay attention to it because as these things get more and more
• 32:31
  complicated and operating more and more at scale and much more in a customer -facing
• 32:38
  capacity, well, we need to understand those interaction dynamics in a much more
• 32:43
  detailed way as we're designing the products.
• 32:47
  Absolutely.
• 32:47
  Well, a lot of work left to do, but a lot of exciting potential.
• 32:52
  And I really enjoyed this conversation.
• 32:54
  And thank you so much for sharing with us the things that you're doing at
• 32:57
  Salesforce.
• 32:58
  my pleasure.
• 32:58
  Thank you for having me.