Latest / The Road to Accountable AI / Lawrence Snapp (TrustScale): Fighting AI Hallucination

Transcript

Lawrence Snapp (TrustScale): Fighting AI Hallucination · The Road to Accountable AI

• 0:09
  Kevin: Hi, I'm Kevin Werbach, professor of legal studies and
• 0:11
  business ethics at the Wharton School of the University of Pennsylvania.
• 0:15
  For decades, I've studied emerging technologies, from
• 0:17
  broadband to blockchain.
• 0:20
  Today, AI is promising to transform our world.
• 0:24
  But AI needs accountability, mechanisms to ensure it's developed and deployed in
• 0:29
  responsible, safe, and trustworthy ways.
• 0:32
  On this podcast, I speak
• 0:33
  music: with the experts leading the charge for accountable AI.
• 0:40
  Kevin: AI systems still make things up, and most people never check.
• 0:44
  My guest is Lawrence Snapp, CEO of TrustScale, which builds tools
• 0:47
  across the AI governance stack.
• 0:50
  We talk about what it looks like to verify AI outputs against evidence,
• 0:54
  why hallucinations won't go away, the prospects for a more robust AI assurance
• 0:58
  ecosystem, why he sees trustworthy AI as analogous to cybersecurity, and why
• 1:04
  he thinks AI users should pause more.
• 1:08
  Here's my conversation with Lawrence Snapp.
• 1:11
  Lawrence, pleasure to have you with me on the podcast.
• 1:14
  Lawrence: It's great to be here.
• 1:16
  Kevin: First of all, uh, why don't you give us some sense of the
• 1:19
  history of TrustScale as it, it grew out of another company called E2F.
• 1:23
  Uh, talk about that, that transition and, and what was the,
• 1:26
  the opportunity that the team saw.
• 1:29
  Lawrence: Yeah.
• 1:29
  TrustScale is actually a DBA for E2F, and so it was a rebranding
• 1:33
  and in, in the spirit of some other things going on in the market, we
• 1:36
  could call it a, a, a refounding.
• 1:38
  Uh, of a company, we're over 23 years old, founded by a French AI scientist,
• 1:44
  uh, who was going to graduate school in France in the '80s when it was kind of
• 1:47
  science fiction to study science, uh, ar- artificial intelligence science.
• 1:52
  Um, but he was truly a, a, a language expert, an AI expert, and he got into
• 1:58
  the business doing French translation, quickly moved into chatbots, voice
• 2:02
  assistants, you can imagine the big three.
• 2:05
  Uh, and from there, a lot of those groups obviously were folded
• 2:08
  into the LLM, uh, data space.
• 2:10
  And so he really evolved very naturally, um, building his company, and he actually
• 2:16
  just retired earlier this year fully.
• 2:18
  But, uh, this was his opportunity to retire and for us to basically bring
• 2:22
  the, the company forward and, and really converge the latest, uh, on the
• 2:27
  technology and the human side to, to help AI become what we all want it to be
• 2:33
  Kevin: Um, so BEJF was doing things like, uh, translation and, and, and data work.
• 2:39
  Um, what was the shift with, with TrustScale in focusing on these AI,
• 2:43
  uh, assurance and governance issues?
• 2:45
  Lawrence: Yeah, I mean, essentially it was the evolution of the industry.
• 2:48
  the company's been doing, we'll call it corpus data, call center, decision science
• 2:53
  in the early days, and then a lot of training and decision science systems,
• 2:57
  for, you know, you can imagine the phone trees that we've all been stuck in.
• 3:00
  and then the early AI agents.
• 3:02
  But this really was this evolution where we followed, the customers.
• 3:06
  Mm-hmm.
• 3:06
  And as the customers moved from, we'll call it simple data to much more advanced
• 3:12
  data, and now obviously AI data, the, the company had a tremendous background
• 3:16
  of their founder's experience- Mm-hmm
• 3:19
  in everything AI, from the corpus pre-training to the post-training to
• 3:22
  the eval to the assurance, and obviously the hot topic is alignment right now.
• 3:27
  Right.
• 3:27
  but he actually had a really deep experience in that, and, so did the team.
• 3:31
  And as the buyers, moved from voice assistance to, to LLM building, they,
• 3:37
  they took us with them, and we were very fortunate to go on that ride
• 3:40
  in the last few years as- Mm-hmm
• 3:41
  you know, we've seen some great growth.
• 3:43
  Kevin: What attracted you to come on board?
• 3:46
  Lawrence: Yeah.
• 3:46
  Listen, I, I think it's, you know, a tremendous opportunity to be at
• 3:50
  the, the frontier- Mm-hmm … of, of possibly one of the biggest humanity
• 3:55
  transformations that has ever occurred.
• 3:57
  so on that side, you know, that's pretty exciting.
• 3:59
  The second part is I- I'm originally from San Jose.
• 4:02
  My internships were at Atari and Apple when I was in high school,
• 4:05
  and for me, you build great stuff, you, you do it responsibly, and
• 4:09
  you hope you change the world.
• 4:10
  And if you do that stuff, usually success follows.
• 4:13
  And, and so I couldn't have seen, a better opportunity to, to try
• 4:17
  to get on the playing field of AI- Mm-hmm from my background.
• 4:20
  I'm also a CPA by trade.
• 4:21
  Mm-hmm.
• 4:21
  So auditing, assurance, controls, systems related to that, is very much in my DNA.
• 4:29
  Kevin: So the term trustworthy AI gets thrown around a lot in different contexts.
• 4:33
  how do you define it, especially with relation to what you do at TrustInAI?
• 4:37
  Lawrence: in, in the, early days of cybersecurity, there was a memo from Bill
• 4:41
  Gates to Microsoft, where I'm an alum as well, called Trustworthy Computing- Mm-hmm
• 4:45
  … and you can still find it online, I think it was 2002, and it really was the, the
• 4:50
  emergence of the whole cybersecurity industry as we know it today.
• 4:53
  But it was about whether or not you could trust a PC.
• 4:55
  I think trustworthy AI is in the same exact camp today.
• 4:59
  We're at the very beginning of a curve of whether or not AI can be trusted.
• 5:04
  and, and it's, it's really following the exact same curve,
• 5:07
  I think, as cybersecurity, although at a much faster pace.
• 5:10
  But it is kind of an amorphous term also, right?
• 5:13
  Yeah.
• 5:13
  And, and that's what makes it so appealing.
• 5:15
  but when you break it down, there's a lot of elements.
• 5:18
  I think in the, the simple level where the world's at now, it's really about
• 5:21
  obviously safety, reliability, and there's a whole hierarchy, a ladder of rungs
• 5:26
  that we work on as far as evaluations and creating rubrics to define it.
• 5:30
  But where is it going is probably more interesting, and that's the nuance.
• 5:34
  The nuance of language, and, and the reality that what I say is maybe
• 5:37
  not what you hear and, and even what you heard might not be what I meant.
• 5:43
  Mm-hmm.
• 5:43
  And so there's a lot going on in the beautiful s- messy humanity
• 5:48
  that we are, that, that goes into this trustworthy AI opportunity.
• 5:53
  Kevin: So all the LLMs still have these things on the box
• 5:57
  that say, you know, it's AI.
• 5:59
  If, you know, may-- don't trust it or it may not be accurate, something like that.
• 6:02
  But you look at the, the technical reports that all the labs put out, and they show
• 6:07
  hallucination rates dropping precipitously and being negligible numbers.
• 6:12
  So, you know, how much is this already a solved problem?
• 6:16
  Lawrence: Yeah.
• 6:16
  listen, I mean, the foundation of this whole thing we have to
• 6:18
  realize is that AI is probabilistic.
• 6:21
  Kevin: Mm-hmm.
• 6:21
  Lawrence: It's a recommendation engine.
• 6:23
  and it's been around a while, spell checkers, grammar checkers in Microsoft
• 6:27
  Word, these things have been around.
• 6:29
  and, and while it certainly has improved, it's still a probabilistic technology.
• 6:33
  It's not deterministic, and it's not conscious.
• 6:35
  But you specifically ask about the, the self-reported benchmarks, of which, you
• 6:40
  know, the-- there, there's a lot of… I call it the, the race of mules right now.
• 6:44
  We're not even in the horse race yet.
• 6:45
  which mule's ahead this week?
• 6:47
  And, I think, I think that end users are smart enough to know, and they've
• 6:50
  certainly experienced it themselves, that, you know, there's a lot of issues left
• 6:54
  in the probabilistic technology that AI is, whether it's recency, whether it's
• 6:58
  accuracy, whether it's faithfulness to the context, or whether it's pragmatic.
• 7:01
  And, it's not just semantics.
• 7:04
  language and, and being able to trust the outputs of a probabilistic system,
• 7:08
  require a lot more than, did you pick the right next word or token?
• 7:13
  so I, I think that the, you know, the, the foundation of this whole thing is
• 7:16
  that you can look at the benchmarks, and especially self-reported, but
• 7:19
  the fox watching the henhouse is not how great businesses were built.
• 7:23
  Mm-hmm.
• 7:23
  And controls are necessary.
• 7:25
  Again, I think it's, it's an amazing superpower.
• 7:28
  We, we believe in it very much.
• 7:30
  But, in order for it to achieve that trustworthy North Star, there are a lot
• 7:34
  of harnesses and controls and systems that need to be built around it, hopefully
• 7:38
  not by the fox watching the henhouse.
• 7:41
  Kevin: Okay, but then if I pivot around to the other side, as you point out,
• 7:45
  this is a probabilistic technology.
• 7:46
  So is it then at some level an unsolvable problem that we can never be certain
• 7:51
  what outputs we're going to get?
• 7:52
  Lawrence: I think this is a little bit like a limit.
• 7:54
  I think we're always gonna be pursuing that trustworthiness.
• 7:58
  Kevin: Mm-hmm.
• 7:59
  Lawrence: there are certain scenarios on the kind of simple questions where it
• 8:01
  probably has exceeded the human ability, to provide intelligence, that is accurate.
• 8:07
  But we all know that, especially in the agentic world that we're in, you know,
• 8:11
  if something gets tripped up in the first agent or sub-agent, every subsequent agent
• 8:16
  is trained to essentially take that string and follow it and, and, and build on that
• 8:21
  confident hallucination that occurred.
• 8:23
  So you can actually end up, in an exponentially different place
• 8:27
  than you thought you would, even though it feels good.
• 8:30
  Mm-hmm.
• 8:30
  And that's how these models are trained.
• 8:31
  So I, I think this is one of those never-ending journeys, and given the
• 8:35
  amount of petabytes of data every day that the world generates that needs to
• 8:39
  go into the AI corpus or post-training, that brings up another issue because
• 8:44
  the amount of slop being generated in those petabytes every day is tremendous,
• 8:48
  which only makes the challenge harder to say, "Hey, was that piece of
• 8:52
  evidence we're comparing the output to actually trustworthy in and of itself?"
• 8:58
  Kevin: What does your company do to address that challenge?
• 9:01
  Lawrence: All right.
• 9:01
  So TrustScale really works across the entire range, of, of AI, from, working
• 9:06
  with, data houses that have proprietary data sets to make sure there's hygiene
• 9:11
  and where there's redundancy or, collisions that they're, they're sorted
• 9:14
  out so that it can involve, really good pre-training of the corpus data.
• 9:18
  and then we do a lot of, post-training as well, not just on the verifiable
• 9:22
  rewards, but certainly on the preferences, as well as the supervised fine-tuning.
• 9:26
  and then you move on and you say, okay, well these rubrics that we created to
• 9:29
  define, either behavior or preferences or outcomes or, or verifiable reliability,
• 9:35
  that, that needs to be evaluated as well.
• 9:36
  So we do evals on a lot of the models.
• 9:39
  and, and then you move it forward, we're actually starting to work with, you know,
• 9:43
  some of my former industry colleagues, in the audit and assurance world,
• 9:47
  looking at how they, as a third party, you know, can have tool sets that they
• 9:51
  can do their independent verification, especially now as regulations are
• 9:54
  starting to come on board to, to protect consumers and businesses from the harms.
• 9:58
  so TrustScale itself works across all of that, and we've got products in each
• 10:03
  one of those stages of the value chain.
• 10:05
  But at the end of the day, what we're, what we're doing in a simple way is taking
• 10:09
  probabilistic outcomes and comparing it to deterministic control plane.
• 10:14
  And that gap analysis is, is not simple.
• 10:17
  This isn't just the baseball score.
• 10:19
  It nuanced.
• 10:20
  And in fact, we call our, our, our business of specifications and rubric
• 10:24
  creation nuance for that reason
• 10:27
  Kevin: Yeah, how do you deal with a situation where there's
• 10:29
  not a, a clear factual answer?
• 10:32
  Lawrence: I think that actually the majority of things in
• 10:34
  life are, are not factual.
• 10:36
  In fact, we don't call ourselves TruthScale, we call
• 10:39
  it Trust Scale for a reason.
• 10:40
  Trust Scale is because, we think that it's up to the individual to,
• 10:45
  to understand what their truth is.
• 10:47
  you know, you might love peanut butter, it might kill me.
• 10:50
  We're both right, those are truths.
• 10:52
  we see a lot of things, especially in the media today, where, you know,
• 10:56
  there's three unemployment rates and, and every politician's using the one
• 10:59
  most favored to themselves, whether it was the whisper number, the first
• 11:02
  reported, or the corrected number.
• 11:04
  Well, that's a good example that all three numbers are true, but how do you trust it?
• 11:08
  So our business is more about empowering humans to make the judgment call,
• 11:13
  or giving a complete and accurate answer when there is nuance, or what
• 11:17
  I call the, the beauty of the, the messiness of humanity, the yellow.
• 11:22
  red and green's real easy, but the yellow is where the magic is.
• 11:25
  Kevin: Hmm.
• 11:27
  Do you have to work with the customers to get them to be able to articulate
• 11:32
  effectively what their preferences are?
• 11:36
  Lawrence: We spend, a tremendous amount of time, and this is where we
• 11:39
  have a lot of in-house experience, and systems for doing it around the
• 11:43
  specification, the rubrics, to measure things, and the quality frameworks.
• 11:47
  some of these are 100-page long, efforts that go into our systems, that then
• 11:53
  can help us actually determine, not just what the right or wrong answer
• 11:56
  is, but when it's not clearly right or wrong, what do you do with it?
• 11:59
  How do you train the model?
• 12:01
  And humans are really complicated things.
• 12:04
  a lot of people will, get excited and say, "Well, that fact is wrong." Well, we're
• 12:08
  more in the empowerment business, I would argue, than the truth-telling business.
• 12:11
  And so our job is to illuminate these nuanced differences and let our customers
• 12:16
  decide how that wants to manifest.
• 12:18
  And hopefully it's in a complete and accurate way so that the end users
• 12:21
  or the agents are fully informed, 'cause that's where I think a
• 12:24
  lot of the world gets in trouble-
• 12:26
  Kevin: What do you mean by that?
• 12:28
  Lawrence: Well, it would be very easy to tell you that
• 12:30
  the unemployment rate was 4%.
• 12:32
  Kevin: Mm-hmm.
• 12:32
  Lawrence: That doesn't mean it's a complete and accurate answer if
• 12:35
  that's what you're asking, because the actual number might have
• 12:37
  been 3.7, and then the corrected number might have been 3.2, right?
• 12:42
  That really changes a lot of judgment.
• 12:43
  So I think that empowering people with complete and accurate information, is
• 12:48
  a much better road to trust than, than trying to say, "Here's the truth."
• 12:53
  Mm-hmm.
• 12:53
  And, and I think as we all know, whether, you know, in any
• 12:56
  relationship and especially if you have kids, you know, th- their truth
• 12:59
  and my truth might be different.
• 13:00
  and, and that's okay.
• 13:02
  And that's where, to be honest, Kevin, it's really exciting, but the future, is
• 13:06
  where we literally have this kind of mass customized or individual perspective of AI
• 13:12
  so that each person can configure the AI to the way that they would trust it most.
• 13:16
  Kevin: If we go down that route though, you mentioned before, auditing and,
• 13:20
  and, and your background, and there's a lot of discussion about moving to,
• 13:24
  you know, evaluations to audits for AI.
• 13:27
  Is that feasible in a world though where it's not-- there's not a right
• 13:31
  answer, it's this, decision based on what's useful for a particular audience?
• 13:35
  Lawrence: Yeah, absolutely.
• 13:36
  But I think a good example, to use the unemployment again.
• 13:38
  If you're auditing the AI output-
• 13:40
  Kevin: Mm-hmm
• 13:41
  Lawrence: you know, you have to wonder about the integrity of the system
• 13:43
  if it's not disclosing the full or complete and accurate information.
• 13:47
  Mm-hmm.
• 13:47
  So completeness would say, "Hey, you know what?
• 13:50
  Y- you, you told the agent it was 4% when it was 4.2.
• 13:54
  Why?
• 13:54
  Because you picked the convenient number, or because you're trying to
• 13:56
  make him feel good?" So I think that audit reports would be much more
• 14:00
  complex than when you have an empirical financial system that you're auditing.
• 14:04
  And, and even those audits though, right?
• 14:06
  If you read the, the opinion letters of these big firms, it's
• 14:09
  always about material correctness.
• 14:11
  Kevin: Mm-hmm.
• 14:12
  Lawrence: I think it's gonna be tough to go beyond materiality, but
• 14:14
  in this case, you know, language is nuanced, and humans want to be
• 14:19
  empowered with more information.
• 14:20
  And so I think we're gonna see very different audit reports in the future.
• 14:23
  In fact, we're developing systems right now that, interpret things, in the yellow.
• 14:28
  You know, the red and green's easy- Mm-hmm … but when it's not clear red
• 14:31
  or green, that's where the magic is, and that's where we spend most of our time.
• 14:36
  Kevin: What do you think are the biggest, gaps today to having a more, extensive and
• 14:40
  broadly functioning AI audit ecosystem?
• 14:43
  Lawrence: Well, the AI audit ecosystem in general is just so nascent, and
• 14:47
  I think part of it is, education.
• 14:49
  You know, it's people like you getting the word out of the, necessity for,
• 14:53
  organizations to be developed that do things in partnership with their customers
• 14:58
  or their clients, but always understanding their North Stars is, I think in AI,
• 15:03
  about humanity, not just profits.
• 15:05
  because this is a little bit bigger than just financial metrics.
• 15:08
  AI has this superpower potential, and we're already seeing, you know,
• 15:13
  little pieces of it here and there.
• 15:15
  But, to unleash this, without continuous monitoring, without proper interpretation
• 15:20
  and reporting and observability on the nuances in between right and wrong, you
• 15:25
  know, it's, it's gonna be difficult.
• 15:26
  So there's a lot of education to be the first and foremost thing
• 15:29
  we need to continue to hammer.
• 15:31
  Then from there, obviously, there will be generally accepted frameworks, that
• 15:35
  will be applied, and I only hope that they're really comprehensive because
• 15:38
  if we're trying to basically mirror humans and, and neuroscience- We, we got
• 15:42
  a long way to go before we get there.
• 15:44
  Kevin: So let's say I'm an enterprise that has an AI governance program,
• 15:47
  help me understand y- where I'm going to most look to Trustio.
• 15:51
  You talked about how you work all the way across the stack, but are
• 15:54
  there particular problems that you're really good at solving for them?
• 15:57
  Lawrence: You know, some people refer to us as an applied governance company.
• 16:00
  Yep.
• 16:00
  And I think the key word is applied.
• 16:02
  Mm-hmm.
• 16:02
  So, it, it is difficult to really try to reach that holy grail of
• 16:06
  trustworthiness- Mm … if, you're not looking at the entire value
• 16:10
  chain of what you're creating.
• 16:11
  And these days, and you talk about enterprise, there's a
• 16:13
  lot of talk about sovereign AI.
• 16:15
  So if you've got your own private corpus and you've got a public corpus-
• 16:18
  Kevin: Mm-hmm
• 16:18
  … Lawrence: are you, conditioning, with hygiene that data before it starts?
• 16:22
  And we, we help with that.
• 16:24
  Mm-hmm.
• 16:24
  Then post-training.
• 16:25
  Preferences.
• 16:25
  Are you giving complete and accurate answers or the shortest ones
• 16:28
  'cause you wanna save server power?
• 16:30
  Well, we can help with that, and we can make sure that you strike that balance,
• 16:33
  of, of value for your organization.
• 16:36
  Then you move forward on assurance, internal audit groups is, some
• 16:39
  organizations that we work with.
• 16:41
  We have tools that we can empower internal audit groups with.
• 16:44
  and that's a, that's a really exciting and emerging area as well.
• 16:47
  So, we're-- we love that one.
• 16:49
  And then the one that's most visible is what we call Argus.
• 16:53
  externally, we released a Chrome extension, and you
• 16:55
  can go to trustargus.ai.
• 16:58
  What's exciting about Trust Argus- Is it essentially real time compares the outputs
• 17:03
  of AI and gives the user red, yellow, green or highlighting with the opportunity
• 17:08
  to correct or adjust the outputs-
• 17:10
  should we find evidence inconsistent to some degree or a great degree
• 17:15
  with what the AI generated.
• 17:17
  Mm-hmm.
• 17:17
  So the most visible thing we have for just people is go to
• 17:21
  Trust Argus and you can try it.
• 17:22
  It's really fun on, on social media and the media, CNN and Fox- and those
• 17:27
  companies too, because it really illuminates where there might be
• 17:30
  contradictory information out there.
• 17:32
  Kevin: how would a company use that?
• 17:32
  I understand that as a demo that you can use it as a customer, but if you're
• 17:36
  an organization, you've got lots of users, lots of AI you're developing, how
• 17:40
  would you actually create the feedback loop, to implement what that tells you?
• 17:44
  Lawrence: The first one- Yeah … is just e- end users, employees.
• 17:46
  Yeah.
• 17:46
  they, they plug it in, whether it's their Chrome browser, Island
• 17:48
  browser, or they actually use our, our chatbot- Yeah Trust Argus, and
• 17:52
  we, we adopt that for enterprises.
• 17:54
  Kevin: Mm-hmm.
• 17:54
  Lawrence: And then that gives a certain layer of protection, right?
• 17:56
  'Cause it detects, it protects through the red, yellow, green
• 17:59
  Kevin: highlighting,
• 18:00
  Lawrence: and it empowers people with evidence so that they can
• 18:03
  correct the AI outputs before they're used in a harmful way.
• 18:06
  So I think it's an exciting tool from an end user perspective, but
• 18:10
  then you asked about the loop.
• 18:11
  Kevin: Mm-hmm.
• 18:12
  Lawrence: The loop is, is so valuable, and that's where the
• 18:14
  post-training opportunity is.
• 18:16
  So everything that's captured in an organization can be looped back and used
• 18:21
  to try to prevent that same hallucination or idiosyncrasy from happening again.
• 18:26
  And so there's two sides to that coin, on Argus.
• 18:29
  The one that end users see to protect them and to empower them to correct
• 18:33
  problems before they cause harm, and then the, the, the administrative side, which
• 18:38
  is those people actually making or, or shaping AI or creating agents- Mm-hmm
• 18:43
  to use as, as input for their training.
• 18:46
  Kevin: And can you drill down a little bit more to, you know, how
• 18:49
  you determine what's red or yellow?
• 18:50
  Presumably, again, the factual issues where it says something
• 18:54
  and you can look at ground truth, that's pretty straightforward.
• 18:56
  But as we talked about, that's not most things.
• 18:58
  So how do you address it?
• 18:59
  Right.
• 19:00
  Lawrence: Yeah.
• 19:00
  I mean, listen, think of this as a, as a real-time high-powered research tool.
• 19:05
  Kevin: Mm-hmm.
• 19:05
  Lawrence: It takes a, atomic level claim, a semantic level, and it will find similar
• 19:11
  information elsewhere, with only about a 15-minute delay worldwide on information.
• 19:15
  And then it will compare it, and then it can get multiple pieces of
• 19:18
  evidence, and enterprises can adopt, their own guidelines if they want.
• 19:23
  let's just say there's three pieces of evidence.
• 19:25
  Two are consistent, one is not.
• 19:27
  Some organizations in certain environments say, "Hey, that's red. I don't even
• 19:31
  want to risk it being yellow." Other organizations, and we'll take a, a
• 19:35
  creative department of, of Disney, right?
• 19:37
  They can just say, "Hey, listen, if two out of three are, are saying that
• 19:41
  that's consistent with other evidence pieces we find that's trustworthy,
• 19:44
  hopefully, then, then that's green.
• 19:47
  So it really depends on the domain.
• 19:49
  and from an enterprise perspective, Argus is completely configurable.
• 19:52
  From an individual perspective, it is not at this point configurable.
• 19:56
  And we essentially, you know, if you take three pieces of evidence, if all
• 19:59
  three are consistent, that's green.
• 20:02
  you know, one or two aren't, it's probably yellow, unless it's a,
• 20:05
  a real, obvious m- m- breach.
• 20:07
  a- and then if all three are wrong, of course that's red.
• 20:10
  but the key is that we give the links to the user to be able to
• 20:13
  do the research themselves, and, and then determine for themselves.
• 20:17
  Again, we want to empower the individual, not take it away from them.
• 20:21
  And so our tools right now, especially in the public facing, stop short of
• 20:25
  actually just overwriting the AI's response, which we could do And that's
• 20:29
  because we really believe trust is about empowerment and knowledge.
• 20:32
  And, and so we're ex- we're excited about how it's working, and we're excited about
• 20:36
  the, the feedback of it so far as well.
• 20:38
  Kevin: Although I assume there's a challenge there, 'cause on the one
• 20:40
  hand, if you push that back to the user, then there's some additional
• 20:45
  friction in the process and some additional time the user has to invest.
• 20:49
  But presumably, if you're in the background rewriting things, you
• 20:53
  have a trust issue about what you're doing, and, you know, the user
• 20:57
  then is no longer having to engage in that, that cognitive activity.
• 21:00
  So how do you find the right balance?
• 21:03
  Lawrence: Well, listen, I, I absolutely love you're going down this path
• 21:05
  because one of our big, tenets as a company is to, to make you pause.
• 21:10
  Mm-hmm.
• 21:10
  Like, we really believe that our job is to help make you pause.
• 21:14
  Why?
• 21:15
  Yeah.
• 21:15
  Because this cognitive surrender has just been consuming,
• 21:18
  especially young people, students.
• 21:20
  and listen, we all went through with social media where it's pretty easy
• 21:23
  to get in, you know, a rabbit hole in Short Horn video or whatever.
• 21:26
  It was cognitively surrendering.
• 21:28
  But with AI, it's really dangerous and much more nuanced, and these systems
• 21:31
  are trained to give you good vibes and, and to actually, the more it knows
• 21:37
  it's hallucinating, make you feel it's even more confident in the response.
• 21:42
  Those are dangerous things.
• 21:43
  So we like, that it prevents cognitive surrender by asking the user to get
• 21:48
  engaged with what they're seeing, and a red and a yellow, those
• 21:51
  are stop signs and yield signs.
• 21:52
  And- Then empowering people with data, yeah, we are putting a little burden
• 21:56
  on them, but I think this is why we're seeing so much in, in like law schools-
• 22:01
  healthcare, people that are in domains where there's real consequences if
• 22:05
  you trust a hallucination blindly.
• 22:07
  but this is, this is the early stages, I think of a 100-year
• 22:10
  transformation of humanity.
• 22:12
  And, and so we're hoping to make people pause, avoid cognitive surrender,
• 22:16
  especially unconsciously, and then, and then get the brain neurons firing again.
• 22:20
  Kevin: And you, you started to get to what I was gonna ask you
• 22:22
  next, which is where, where are you seeing the most interest and demand?
• 22:25
  What kinds of companies?
• 22:26
  Lawrence: Yeah.
• 22:26
  So o- obviously, the, the higher risk domains.
• 22:29
  I think that there's, you know, plenty in law and healthcare, but academia
• 22:33
  across the board, every discipline.
• 22:35
  and then you move down, I think that the other thing is the, is the content houses
• 22:39
  that have repositories of private data that are, you know, trying to figure out
• 22:43
  like, "Hey, I don't want to sell this to an AI frontier lab and then have my
• 22:47
  data show up as mine, but misinterpreted or taken out of context or, or not
• 22:53
  pragmatic relative to the bigger picture."
• 22:55
  Because data alone is, is insufficient.
• 22:59
  It is about the semantics, the contextual, and the pragmatic level,
• 23:02
  and all of it needs to be considered.
• 23:04
  And, and that's what our Trust Scale Engine does behind Argus on the visible
• 23:08
  side and, and Argus for the enterprise.
• 23:11
  Kevin: Mm-hmm.
• 23:12
  And what happens about this idea of the user taking control and pausing
• 23:17
  when we're in a world of agents and the actions are often not taken by humans?
• 23:23
  Lawrence: Yeah.
• 23:24
  Well, this is where it trips into what we have is really a
• 23:26
  cybersecurity product, right?
• 23:28
  Yeah.
• 23:28
  Because these nuances, hallucinations or some people like to call them
• 23:32
  confabulations, these are threat vectors.
• 23:34
  And one of the big things is that when an agent is, integrated with Argus,
• 23:39
  it could essentially force the agent to pause and get a human to say,
• 23:41
  "Hey, should I keep going or not?"
• 23:44
  And so we think that this is a cybersecurity product.
• 23:47
  We think it's a hallucination firewall.
• 23:49
  These are terms that our customers use- Mm when describing what
• 23:53
  they're doing with our product.
• 23:54
  and, and we're excited about that.
• 23:56
  So I think that the, the, the autonomous agent world, you know, there, there's
• 24:01
  gonna have to be gates and, and there's gotta be systems to put those gates up
• 24:06
  because no human can keep up with the volume that AI is producing these days.
• 24:11
  Kevin: Mm-hmm.
• 24:12
  It's interesting you frame it that way, 'cause I mean, I think of the
• 24:14
  cybersecurity, there's an attacker out there, and certainly we, you know,
• 24:17
  we have prompt injection and data poisoning and things like that where
• 24:21
  there, there's some deliberate act that's, that's causing the, the, the
• 24:24
  incorrect information with the AI.
• 24:25
  But if it's, if it's a hallucination of something internally coming in the model,
• 24:29
  it seems to me like that's different.
• 24:31
  Lawrence: These models are trained to behave a certain way, to make
• 24:35
  the user feel a certain way.
• 24:37
  Kevin: Mm-hmm.
• 24:38
  Lawrence: Some models are trained to flat out just make it up.
• 24:41
  Mm-hmm.
• 24:41
  Some are trained to say, "Hey, I'm not exactly sure. Check this." But that's
• 24:46
  rare right now because that would really get in the way of people paying for it.
• 24:50
  a- and then there's, there's others where, you know, we're seeing, you
• 24:53
  know, 70, 80%, according to one Stanford study, hallucination rates
• 24:58
  on, on like a, a legal dummy- Yeah
• 25:00
  on certain models.
• 25:01
  So there, it's all over the map, but these things are trained to give you
• 25:05
  good vibes and hallucinate deliberately.
• 25:07
  Again, it's a recommendation engine.
• 25:08
  It's not a deterministic outcome.
• 25:11
  Yeah.
• 25:11
  And we need to, always kind of consider that, but it's very easy to lull ourselves
• 25:16
  to sleep and just use the superpowers
• 25:18
  Kevin: Yeah, I guess what I was wondering about is it's, you know, just
• 25:21
  the, the framing of it, and it's, it's interesting if you're getting positive,
• 25:24
  responses from, from customers on, on talking about in cybersecurity terms.
• 25:29
  Just, you know, still that, that's not a situation where the AI company is
• 25:32
  acting in a deliberately hostile way.
• 25:34
  As you say, it's, it's intrinsic in the model, and it's in line with
• 25:37
  their, their business interests.
• 25:40
  but, you know, I'm just, I'm just, you know, curious and interested
• 25:43
  about how that seems to resonate, the talking about cybersecurity.
• 25:48
  Lawrence: Yeah.
• 25:48
  So, listen, hallucinations, depending on the domain, are threat vectors.
• 25:52
  Yeah.
• 25:52
  They can kill people.
• 25:53
  Yeah.
• 25:53
  they can hurt people.
• 25:54
  They can harm people.
• 25:55
  They can harm communities.
• 25:56
  these things are real, and there's a lot of gravity.
• 25:58
  And luckily, and, and obviously more recently with some of the press, th-
• 26:02
  there's been some more seriousness, Yeah
• 26:04
  around it.
• 26:05
  But the problem is that, that, you know, we know there's needles in this haystack,
• 26:09
  we don't know what they look like.
• 26:11
  And, you know, as a human, you'd have to check every word- Mm-hmm … to
• 26:14
  see if that's where the needle was.
• 26:16
  Well, that's just not possible.
• 26:17
  but those needles can hurt, and, and that's why this cybersecurity framing is
• 26:22
  it just kind of became natural for us.
• 26:24
  And, and our customers and our prospects look at it and say, "Wow, I could sure use
• 26:28
  some hallucination protection." Mm-hmm.
• 26:30
  Now, there's others that call this, "Well, these are just
• 26:32
  confabulations like humans do." Yeah.
• 26:34
  And, and that's a softening of it.
• 26:36
  Mm-hmm.
• 26:36
  I'm not saying that every employee is perfect or remembers things perfectly.
• 26:41
  and yes, we as humans confabulate, that, that's for sure.
• 26:44
  we might, might each remember this discussion slightly different.
• 26:47
  That's normal.
• 26:48
  but these models, when they're pulling data and then confabulating
• 26:52
  or hallucinating in high-risk domains, that, that's something that needs to
• 26:56
  be taken very seriously, and that, that's where we're excited that our, our
• 27:00
  services and solutions really, can help protect, detect, protect, and correct
• 27:05
  the problems before they cause harm.
• 27:07
  Kevin: Right now, there's a lot of attention being paid to
• 27:10
  the sort of catastrophic risk potentially of the frontier models.
• 27:15
  a- as you say, wh- when you're talking about hallucinations, there can be
• 27:18
  very serious consequences, including loss of life, but it's, it's not at the
• 27:22
  scale of, these kinds of systemic risks that we're, that we're talking about.
• 27:27
  do you feel like all of the attention being paid to these
• 27:30
  potential existential threats is, is distracting people from this other
• 27:35
  domain that you're involved with?
• 27:37
  Lawrence: Well, it, it's interesting.
• 27:38
  I, I would argue that even some of these existential threats, that are, that are
• 27:42
  headlining the last few weeks, might start as a simple small hallucination.
• 27:47
  Kevin: Mm.
• 27:47
  Lawrence: and so you can't quite ignore, No … the little stuff if
• 27:50
  you're worried about the big stuff.
• 27:51
  Now, I think taking the knob and turning it all the way right is a great way
• 27:55
  to get PR and the- Mm-hmm … mass audience globally- Yeah … to,
• 27:58
  to care and action hopefully.
• 28:00
  But I'll tell you, we have seen more times than not, one small hallucination
• 28:06
  have a ripple effect that cascades to create exponential, drift or
• 28:10
  variance from the objective originally.
• 28:12
  And like, why did that happen, and where did that happen?
• 28:14
  Well, more often than not, if you peel the onion back and you find the, the,
• 28:18
  the initial source or the core of that, it started with something small that
• 28:22
  became very big very fast, and these are the, the existential threats, right?
• 28:26
  The bot's gone wild.
• 28:27
  and so there's a, th- there's a, there's a whole host of, opportunity for
• 28:31
  companies like ours to figure out, how to, how to protect, you know, and, and
• 28:35
  detect and protect against the little stuff so it doesn't become big stuff.
• 28:39
  but the big players, you know, you can, you, you can, you know,
• 28:42
  flash the shiny object of, "Oh, we're all gonna kill ourselves,"
• 28:45
  a- and you get plenty of headlines.
• 28:46
  it's really hard to say that one problem is, is any greater
• 28:50
  or, or less than the next.
• 28:52
  Kevin: Absolutely.
• 28:53
  Lawrence, thank you so much for your time.
• 28:55
  I appreciate the opportunity to hear about what TrustScale is doing.
• 28:58
  Lawrence: Kevin, thank you very much.
• 29:02
  This has been The Road to Accountable AI.
• 29:04
  Kevin: If you want to go deeper on AI governance, trust, and responsibility
• 29:07
  with me and other distinguished faculty of the world's top business
• 29:11
  school, sign up for the next cohort of Wharton's Strategies for Accountable
• 29:14
  AI online executive education program, featuring live interaction
• 29:19
  with faculty, expert interviews, and custom designed asynchronous content.
• 29:23
  Visit execed.wharton.upen.edu/acai for full details.
• 29:31
  I hope to see you there