Latest / The FIT4Privacy Podcast | Exploring ISO Standards AI Privacy | Grow Skills Store / How to Evaluate AI You Can Trust E166

Transcript

How to Evaluate AI You Can Trust E166 · The FIT4Privacy Podcast | Exploring ISO Standards AI Privacy | Grow Skills Store

• 0:00
  AI is machine that are good at making guesses based on data.
• 0:08
  In fact, this is the answer. So I was doing the AI Literacy
• 0:12
  for Kids programme in January at my local library and this is the
• 0:17
  this is the definition I gave to the kids in AI evaluations.
• 0:21
  What we are trying to do is we're trying to use different
• 0:25
  techniques, different methods or methodologies to catch those
• 0:31
  issues and also to understand the behaviour patterns of AI for
• 0:37
  traditional software. When we do testing, we're
• 0:40
  testing the logic written in the code.
• 0:44
  Hi, my name is Punit Patia and you're listening to Fit for
• 0:47
  Privacy podcast where AI meets privacy.
• 0:50
  This is not legal advice. Thanks for listening.
• 0:54
  How do you evaluate AI? Yes, we talk about AI being
• 0:58
  responsible, AI being compliant, AI being relevant, AI being
• 1:03
  private, AI being ethical and so on.
• 1:06
  But how do you make sure? How do you evaluate that?
• 1:09
  AI is compliant, AI is good enough, AI is fit for purpose?
• 1:13
  Well, that's a very interesting question.
• 1:15
  And for that, we have a very interesting guest, Stella Liu,
• 1:19
  who's going to guide us on how to evaluate AI.
• 1:22
  And as we discussed that with her, do take advantage of the
• 1:26
  fit for privacy consulting and visit Grow Skills dot store and
• 1:32
  take a look at ISO related trainings, EU regulation related
• 1:36
  training. We talk about AI.
• 1:38
  You can learn about ISO 42,001. You can be a certified AI
• 1:42
  professional, you can be a certified AI risk manager and
• 1:46
  also visit our new website www.fitforprivacy.com.
• 1:51
  For now, let's go and talk to Stella.
• 1:54
  Hello and welcome to another episode of the Fit for Privacy
• 1:56
  podcast. And today we are with Stella
• 1:58
  Liu. Stella, welcome to Fit for
• 2:00
  Privacy podcast. Thank you for having me.
• 2:04
  It's a pleasure to have you and I'd like to start with a very
• 2:07
  basic question. How would you define AI?
• 2:11
  Because you do so much work in AI.
• 2:13
  But I like to see in a few words, how do you define AI?
• 2:17
  Yeah, I have a very high level answer for that.
• 2:21
  So AI is machine that are good at making guesses based on data.
• 2:30
  In fact, this is the answer. So I was doing the AI Literacy
• 2:35
  for Kids programme in January at my local library and this is the
• 2:40
  this is the definition I gave to the kids.
• 2:43
  That's a good one, the machine that makes kisses.
• 2:46
  And I think in future people expect a lot of things from
• 2:50
  them. But that's in future, not now.
• 2:52
  And that future is what the media scares us about.
• 2:55
  There are lots of challenges that are associated with AI,
• 2:59
  Yeah. And the challenge, one of the
• 3:01
  challenges companies are facing or people are facing is how do
• 3:06
  you identify or evaluate if it's trustworthy AI, responsible AI,
• 3:12
  ethical AI, or in the legal partners we say compliant AI.
• 3:16
  So do you have methodology guidance on how can anybody, if
• 3:23
  they want to check a certain compliance, maybe say with the
• 3:26
  EUAI Act, maybe say with ISO 42,001 or NIST AI framework or
• 3:31
  any of the other responsible AI frameworks, how would they go
• 3:34
  about it? What should they start with?
• 3:37
  Yeah. So this is definitely a
• 3:39
  challenge that we all face in the industry today.
• 3:43
  Again, like going back to the definition I just gave, because
• 3:47
  AI is machine making gases, it means it is non deterministic.
• 3:53
  And what that entails is it can sometimes it makes good gases,
• 3:58
  but occasionally it makes bad gases.
• 4:01
  And in AI evaluations, what we are trying to do is we're trying
• 4:05
  to use different techniques, different methods or
• 4:10
  methodologies to catch those issues and also to understand
• 4:18
  the behaviour patterns of AI before it goes into release to
• 4:24
  help us and doing all that to help us understand and predict
• 4:28
  what AI will perform in production.
• 4:31
  So I would say that is what we try to do with AI evaluations.
• 4:36
  And this field, AI Eval is still very new.
• 4:40
  So a lot of the practises are still taking shape There are I
• 4:44
  wouldn't say there is a very clear consensus on best
• 4:50
  practises yet again, because it is very new, but from since like
• 4:55
  2022, since the since the introduction of ChatGPT.
• 5:00
  I have been working in this field for several years now and
• 5:03
  we from my experience and from the work that I have done in
• 5:07
  this field, there are some like best practises and practise
• 5:11
  practical tips that I share with the community with people who
• 5:16
  are working on AI emails. And you mentioned like for
• 5:22
  compliance and that is definitely a big part of AI
• 5:25
  emails today. And besides, like, besides
• 5:29
  compliance work guardrails in AI emails that we check for,
• 5:33
  there's also a lot of evaluations that we do to, to,
• 5:40
  to try to assess the quality of the product.
• 5:45
  So for example, if we have a customer support bot for a, say,
• 5:51
  a insurance company, then what is this chat bot is expected to
• 5:56
  do? All that should be defined in
• 5:59
  the product requirements document, not just what it is
• 6:03
  supposed to do and what it is not supposed to do.
• 6:07
  And all those should be defined in the product requirements
• 6:10
  document. And for us who work in AI evals,
• 6:15
  our role is to work closely with the product team to translate
• 6:19
  that into quantitative and also repeatable measures, again to
• 6:24
  help us predict how the product is going to perform in
• 6:27
  production in the real world. So essentially what I understand
• 6:31
  is AI evaluation is not just about post product deployment,
• 6:36
  it's also about working with the teams of the design and
• 6:40
  requirements phase and being ready to demonstrate everything
• 6:45
  that you need in the end. So it's not a post check, it's a
• 6:48
  pre engagement, pre conversation that you have.
• 6:51
  Yes, and that is a great point. I would say there are three
• 6:55
  different use cases that I see of AI evals during the product
• 7:00
  development cycle. 1 is what I call evaluation driven
• 7:05
  development and that probably raised a bell has very similar
• 7:08
  to what we are familiar with test driven development and it
• 7:13
  is the same concept. So like why we have test driven
• 7:17
  development for traditional software, we need evaluation
• 7:22
  driven development for for AI products because AI products
• 7:28
  again because it is non deterministic, it is a checking
• 7:33
  or understanding the behaviour pattern, it can be extremely
• 7:37
  challenging. So we use evaluation metrics as
• 7:42
  our compass during the product development cycle from iteration
• 7:48
  to iterations to help us identify where we should
• 7:52
  improve, where where we should adjust or update the design.
• 7:56
  So that is evaluation driven development.
• 7:59
  I would say AI evals is also, and this is probably the most
• 8:03
  common use case for AI Evals is right before release.
• 8:07
  We want to run the series of cheques just like what we do in
• 8:11
  software testing. We do AI evaluations to help us
• 8:15
  make release decisions. So basically it helps us to
• 8:19
  answer the question of whether this product is ready for
• 8:21
  release. So that is the second use case.
• 8:24
  And then once the product goes live in production, we still run
• 8:31
  AI evals as like CICD cheques and also as like regression
• 8:37
  tests whenever we need to make changes or make updates to the
• 8:41
  products where whenever we, for example, want to switch to the
• 8:44
  latest model from our model provider.
• 8:49
  And now evaluating AI can mean multiple things for people.
• 8:53
  If you, if I talk to you, it'll be different.
• 8:57
  If I talk to some tester, it's all about testing.
• 9:01
  That's what they consider AI evaluation.
• 9:03
  If I talk to a compliance colleague, he or she would say
• 9:06
  it's about controls or compliance with policy.
• 9:10
  So is all of that AI evaluation or do you define it slightly in
• 9:15
  a a separate way? Yes.
• 9:18
  So in my definition, I would say all this are AI evaluations and
• 9:22
  I come from a background of applied science and data
• 9:26
  science. So to me maybe I'm more looking
• 9:30
  from the angle of what methodology or what technique we
• 9:34
  use. So in that sense all this are AI
• 9:37
  evals, which I I interpreted as using scenario based testing to
• 9:45
  probe the products behaviours. And it has different, it can be
• 9:51
  used for different types of evaluations.
• 9:53
  And I think that goes back to the different, different, how do
• 9:58
  I call it like a different purpose of evaluation like for
• 10:01
  compliance or for quality cheques?
• 10:06
  That's right. But now earlier when we were
• 10:10
  developing machines, as you said, machines that can guess.
• 10:14
  So when we were just building machines or software, it was all
• 10:18
  about software testing. Right.
• 10:21
  Nowadays and of course then came the word compliance, privacy and
• 10:25
  so on and so forth. And then came governance, data
• 10:28
  management and so forth. But with AI we are extensively
• 10:31
  talking about, it's very challenging to evaluate if it's
• 10:35
  responsible, it's trustworthy, it will do the right thing, if
• 10:39
  it is not biassed, if it is fair and all those things.
• 10:43
  Now, why in your view is it's so complex with the same IT
• 10:47
  software but it's a little bit more advanced?
• 10:50
  Yes, and that is a great question.
• 10:54
  The gap that I see in today's practise from many teams is that
• 11:00
  it requires doing AI evals, requires a mental model shift
• 11:04
  from developing tests or testing plans for traditional software.
• 11:09
  So like for traditional software, when we do testing,
• 11:14
  we're testing the logic written in the code.
• 11:18
  So we're testing whether the code is written correctly.
• 11:22
  If I if, if I simplify it. But in AI evals, of course, we
• 11:28
  still need to do like for AI systems, we still need to test
• 11:32
  the, the like the code itself. But what it what differs from
• 11:36
  traditional software is that with AI systems today, we're
• 11:40
  also again like probing the behaviour of AI models.
• 11:45
  So it may be just one line in the code that like now call this
• 11:50
  model and get inference. And that one line has no logical
• 11:56
  flaws. But the model itself, because of
• 11:59
  the non deterministic nature of generative AI, we need to test
• 12:06
  it. We need to test its behaviour
• 12:09
  pattern and your different scenarios with.
• 12:13
  Maybe I can give a simple example.
• 12:15
  Say I ask a chat bot and again, like I use chat bot as an as an
• 12:21
  example here, but AIA fails is not only for tax only
• 12:25
  applications, It also works for it also very much needed for
• 12:30
  multimodal yeah applications. But again, coming back to the
• 12:33
  customer support chat bot example, I can ask it say I want
• 12:39
  to track the status of my order for on this e-commerce website.
• 12:47
  I can ask the the the question very in a very straightforward
• 12:50
  fashion. And I expect this chat bot to to
• 12:54
  give me an answer, give me a correct answer.
• 12:58
  But in AI evals only testing with that only one scenarios is
• 13:03
  not enough. I also want to test this
• 13:07
  question but with slightly different phrasing or framing.
• 13:12
  And I want to test that if with like different variations of the
• 13:16
  sick question is the AI chat bot is still able to give me the
• 13:20
  correct answer. And again, like this is this is
• 13:23
  one of the simple examples with AI evaluations that you can
• 13:29
  imagine with different user paths, with different use cases,
• 13:35
  there can be many different scenarios that we want to and we
• 13:41
  should test with AI systems and AI products.
• 13:46
  And I would also like to add that it is challenging because
• 13:50
  it is almost guaranteed that we will never get 100 out of 100
• 13:56
  because again, AI is non deterministic.
• 14:00
  Yeah, when we talk about evaluations or evals, I read
• 14:07
  somewhere wherein you posted a few weeks ago saying evaluations
• 14:12
  without analytics means you're guessing what users actually
• 14:15
  experience and analytics without any evaluation means you're
• 14:20
  debugging. In production, you need both and
• 14:23
  most teams have won. Would you like to elaborate on
• 14:27
  that concept that you posted a few I think or talked about a
• 14:30
  few weeks ago and of course you talk about in your courses?
• 14:34
  Yes, and thank you for bringing that up.
• 14:36
  That is what I have been advocating for in the in the AI
• 14:40
  community. I would say in a little
• 14:44
  background of that, I would say before 2025 very few people were
• 14:49
  talking about AI emails. We were all in the hype of AI,
• 14:53
  We're still in the hype of AI. But at that time, I think it was
• 14:56
  mostly let's, let's just build it and ship it without doing a
• 15:01
  lot of cheques. And then I would say in 2025,
• 15:05
  more and more people realise the importance of AI evals and many
• 15:09
  more teams are doing that. Switch is great, but the I'm
• 15:14
  I've been advocating for don't just rely only on evals, but
• 15:18
  also doing analytics. And the reason for that, I have
• 15:23
  analogy for it. So testing artificial
• 15:27
  intelligence is actually surprisingly similar to how we
• 15:31
  test human intelligence. So how do we test human
• 15:34
  intelligence in schools? We design all those like
• 15:38
  standard tasks like GRE or like SAT tasks for students.
• 15:45
  And those standard tasks are actually very similar to AI
• 15:50
  evaluations in concept, where with like if we think about test
• 15:56
  for human for students, each, each test is like a set of
• 16:03
  scenarios where test set in AI emails.
• 16:07
  And we ask the human, we ask the student to answer to each one of
• 16:10
  them and we check to see if they, if they understand the
• 16:15
  concept, if they, they are able to give correct answers.
• 16:21
  But all that standard task is actually a proxy to what this
• 16:26
  student or how the student would perform in real world tasks.
• 16:33
  And to me analytics is like the performance review we do when we
• 16:38
  enter corporate, when we are doing like real world jobs for
• 16:44
  real world tasks, the AI evals before most because most of the
• 16:50
  AI evals is down pre release. So all those tasks, all those
• 16:55
  standard tasks that we develop is our best guess of what this
• 17:01
  products would encounter in production or in real world.
• 17:06
  And that is again it reflects the product teams understanding
• 17:15
  of what the user needs. But as we all know that is
• 17:20
  different from what how users use our product for.
• 17:24
  This is also true for even like non AI products.
• 17:28
  So analytics is the monitoring and feedback system in
• 17:32
  production. Now we have users using this
• 17:34
  system. Now we have seen how users are
• 17:37
  using it and there must be like edge cases where like different
• 17:41
  user paths that we didn't expect before release.
• 17:44
  And now with all that information, let's check and see
• 17:47
  how our product behaves in production.
• 17:50
  And let's make iterative updates to our product based on that
• 17:56
  real world, real user feedback, using analytics of course.
• 18:01
  OK, that makes sense. Now I have no of course it is.
• 18:06
  It's very clear. And the example of a class
• 18:09
  evaluation makes sense for the analogy.
• 18:12
  But I'm tempted to ask, in all these evaluations, what's your
• 18:16
  view on using synthetic data? Because you have data which can
• 18:21
  be biassed, but you can create synthetic data which probably or
• 18:24
  hopefully is not biassed. So what are you an advocate of
• 18:27
  using synthetic data or are you against it?
• 18:29
  What's your view? Excellent question.
• 18:33
  I also posted a lot about that on LinkedIn and also in myself
• 18:36
  Stack newsletter. There are, I would, I would say
• 18:40
  there are, I would say it depends.
• 18:44
  So there are scenarios where cases where we are building the
• 18:52
  product to solve an existing issue or existing challenge.
• 18:59
  An example of that would be again like using our customer
• 19:02
  support bots example. Let's say we want to build this
• 19:05
  customer support chat bots for this e-commerce company.
• 19:10
  And before we build all this product, we actually already
• 19:13
  have a call centre at the company and we're trying to use
• 19:20
  this chat bot to reduce the workload for the call centre.
• 19:24
  If that is the case, then we already have reached data
• 19:27
  collected in the past that we can use to test the chat bot.
• 19:32
  I can use the for example, like the incoming inquiries or I can
• 19:37
  even use the call centre transcripts and I can extract
• 19:43
  different scenarios and user queries or inquiries and use
• 19:47
  that to build the test set for for the chat bot.
• 19:53
  So in that case, this is not synthetic data.
• 19:56
  It's not. It is based in true true user
• 20:01
  data. So I would say for use cases
• 20:04
  similar to this, if you already have data collected then
• 20:07
  definitely use that, use that seed data instead of using
• 20:11
  synthetic data. But there are also cases where
• 20:15
  we are Co starting meaning we don't have data collected from
• 20:19
  the past. Then in those cases we will have
• 20:23
  to rely on synthetic data. But I would say we should also
• 20:30
  we, we should not blindly rely on synthetic data.
• 20:34
  And I will talk about why in a minute.
• 20:37
  The my recommended approach to costar problem is that work with
• 20:44
  the product team closely with the product manager and the UX
• 20:48
  and also the stakeholders to map out the critical user paths of
• 20:53
  the product. And for each user path, what is
• 20:57
  the what we call like if it is in in software testing, we call
• 21:03
  like normal case which is the happy path and we also have
• 21:07
  alternate path which is the unhappy paths.
• 21:10
  So start from there, map it down.
• 21:14
  And then once we have that list of different scenarios, we can
• 21:19
  use AI, we can use LLM to augment from that and then we
• 21:24
  can build a synthetic data set from there.
• 21:28
  So although we're still relying on AI on LM to generate this
• 21:32
  exact data, but there is it is grounded in the process design
• 21:38
  and grounded in like don't for example, domain knowledge from
• 21:42
  stakeholders for subject matter experts and use that as a much
• 21:48
  more responsible and reliable reliable way of building test
• 21:53
  set as opposed to just like PROMPT and LLM that asked it to
• 21:57
  generate a synthetic data set. And the problem with over
• 22:04
  relying on synthetic data is because we have this what we
• 22:11
  call preference leakage problem with synthetic data.
• 22:17
  And an example of that is, for example, I'm using a model from
• 22:22
  Open AI for my AI product. And I use, let's say I use
• 22:29
  another model word, the same model from Open AI to generate
• 22:35
  the test set. Then I'm almost guaranteed to
• 22:40
  get a good score from that testing because of this like
• 22:44
  preference leakage issue that has been discussed a lot in in
• 22:49
  research community academia. That's an interesting way of
• 22:54
  looking at it, because I usually look at it anything that's
• 22:59
  generated artificially synthetic data.
• 23:01
  But you seem to make a difference between when you
• 23:04
  don't have any baseline data and you generate data that's pure
• 23:07
  synthetic data. But when you have customer data
• 23:11
  and maybe you want to protect that data and you don't want to
• 23:14
  use that production data into testing, you generally use that
• 23:17
  to synthetically generate data. You differentiate it slightly.
• 23:21
  You don't call it pure synthetic data if I understood well.
• 23:25
  Yes, yes, we all have our definitions slightly
• 23:29
  differently. The difference I draw there is
• 23:32
  whether there is new information coming into that data generation
• 23:37
  process. Where are we just relying on the
• 23:39
  model to generate synthetic data?
• 23:42
  Yeah. But it's good to have that
• 23:43
  bifurcation for the segregation. So you talked about being
• 23:48
  responsible and being responsible also has two or many
• 23:51
  other dimensions, but 2 broad dimensions of protecting privacy
• 23:55
  and also being ethical. So in the AI events that you do,
• 23:59
  how do you take care of these two dimensions, being ethical
• 24:02
  and being respectful for privacy of individuals?
• 24:06
  Yeah. So when we talk about like
• 24:10
  ethical use of AII would say the first few evaluation metrics
• 24:16
  that come to mind are bias and also fairness.
• 24:22
  In my definition, that would be some examples of the key metrics
• 24:26
  that we use to test ethical use of AI.
• 24:31
  And there are many different ways to test bias and fairness.
• 24:35
  And we also talk a lot about like privacy with AI.
• 24:40
  And one big topic in that space is PIL leakage with AI products.
• 24:49
  And do you then look at just privacy by principles
• 24:55
  perspective or also going to the dimension of legal compliance
• 24:58
  with say the GDPR, the CCP or other dimensions?
• 25:04
  For the applications that I developed, yes, I also looked
• 25:07
  into like the regulations and the policies and that has been
• 25:11
  the challenging part not just for me, but for everyone who is
• 25:14
  working on AI evals today. I, I live in the US and the
• 25:21
  applications I develop or evaluate are for mostly users in
• 25:27
  the USI would say right now at the federal level, we haven't
• 25:34
  had a lot of like guidance on like privacy or regulation or
• 25:38
  compliance for AI products. There has been where there have
• 25:44
  been more state level initiatives try to lay out like
• 25:51
  the guided guidelines where regulations where compliance
• 25:54
  requirements for AI products. We have an AI act from Colorado,
• 26:00
  we have an AI act from California.
• 26:03
  And also interestingly, although we don't have a federal
• 26:07
  guidelines, the current administration they issued, I
• 26:11
  forgot the exact name of that document, but basically that it
• 26:17
  says that for for federal government procurement, it
• 26:23
  requires the vendors to to for example, like submit their
• 26:29
  evaluation results as part of the procurement requirements.
• 26:36
  So I would say that is on the regulation side of things.
• 26:40
  But even without clear signals or guidelines, every company is
• 26:45
  also doing their own regulation cheques.
• 26:48
  I work in education tech, so a lot of the applications we
• 26:52
  develop are for students and there has already been policies
• 26:59
  or regulations for student facing applications for for
• 27:04
  education for use cases in education.
• 27:07
  So in the US Furpaw is a big one.
• 27:10
  And we also have like different policies at at universities.
• 27:17
  I can give an example. Well, maybe taking one step back
• 27:22
  for Inferpa Pi is also an important, it's also a focus and
• 27:28
  we do a lot of that. But in addition to that, there
• 27:33
  can also be policies like for example, in a, in a, in a
• 27:39
  educational setting, If I developed a chat bot that is
• 27:44
  student facing, and if I see student expressing, for example,
• 27:50
  suicidal thoughts when interacting with this chat bot,
• 27:53
  what do we do? What are the responsibilities
• 27:57
  not just for the university, but for admins?
• 28:00
  What are our responsibilities? So a lot of that again, this has
• 28:04
  been a challenge for, for for everyone in AI community because
• 28:11
  it is hard to generalise. So in when I work on that, I go,
• 28:19
  I basically go through all the policies at the organisation and
• 28:24
  I check how we can quantitize each one of that and whether
• 28:30
  each, for example, like prohibited use can be translated
• 28:34
  into an evaluation metric. And that is not something that
• 28:39
  you would expect a applied scientist to do.
• 28:41
  But that is what we are doing today in order to do AI emails,
• 28:44
  right? Oh yeah, that's interesting.
• 28:48
  Now coincidentally, AI Evals is also the name of the brand of
• 28:54
  the company that you are associated with.
• 28:57
  So tell us about AI Evals more as a company rather than AI
• 29:01
  Evals as a process or activity. What does AI?
• 29:05
  What is AI Evals? What is it about?
• 29:07
  Who does it serve and what's your mission?
• 29:10
  Yes. So myself, I have been working
• 29:14
  on AI Evals since 2023 and starting from last year, I
• 29:18
  founded this organisation called AI Evals and Analytics.
• 29:23
  And the goal of this organisation is to set up the
• 29:29
  best practises in this space and also provide training to
• 29:34
  individuals and consulting services to companies who need
• 29:38
  to do AI Evals for whether it's their internal AI products or
• 29:44
  help them to assess external AI vendor solutions.
• 29:49
  That's good to know. And now if anyone based on this
• 29:54
  conversation that we had wants to talk about AI evals either in
• 29:59
  context of your work or in general, or maybe wants to reach
• 30:03
  out to you, what's the best possible way?
• 30:06
  Yeah, I would say the best way to find me would be lichting.
• 30:10
  So my name, yeah. So search Stella Wenxin Liu on
• 30:15
  LinkedIn and you can also check out our course on Maven through
• 30:20
  ai-evals.org. That is AI-E Val s.org.
• 30:29
  That's very nice of you to share your thoughts.
• 30:32
  And I didn't realise we, we have completed the time also and we
• 30:37
  have had a very good conversation.
• 30:39
  So Stella, thank you so much for your time.
• 30:42
  It was wonderful to have you. Thank you, it was great chatting
• 30:46
  with you. Thank you for joining us on the
• 30:49
  Fit for Privacy podcast. We hope you found today's
• 30:52
  conversation valuable and thought provoking.
• 30:54
  Subscribe for more insights like this.
• 30:56
  If you enjoyed this episode, we'd appreciate that you'd share
• 30:59
  it and write us a review. For consulting, visit
• 31:02
  www.fitforprivacy.com. For certifications and training
• 31:07
  on ISO standards, visit www.growskills.store.
• 31:12
  You can also contact us for bespoke training for your staff
• 31:15
  and management. Until next time, stay safe, stay
• 31:18
  compliant, and grow your skills.