Latest / The Road to Accountable AI / Wendy Gonzalez: Managing the Humans in the AI Loop

Transcript

Wendy Gonzalez: Managing the Humans in the AI Loop · The Road to Accountable AI

• 0:00
  Wendy, thank you for joining me on the road to Accountable AI.
• 0:04
  Very glad to be here, Kevin.
• 0:07
  First, tell us a little bit about what Sama is and what role the company plays in the AI development process.
• 0:16
  Sama is a data annotation and model valuation company.
• 0:19
  So we basically take care of structuring, validating, or fine tuning the data that goes into training and fine tuning AI models, whether it's traditional ML models or a GenAI.
• 0:34
  And where did it come from?
• 0:36
  How has it developed over the years?
• 0:39
  We have a bit of an interesting origin story.
• 0:41
  So we actually started as a nonprofit back in 2008, really focused very specifically on a social mission of lifting people sustainably out of poverty by providing them jobs,
• 0:54
  full-time jobs in the digital economy.
• 0:56
  So we work in underserved communities.
• 0:58
  We are working to provide first-time entry-level jobs in tech and full-time jobs, which are scarce.
• 1:06
  So six out of seven people in Sub-Saharan Africa are in the informal economy.
• 1:10
  And our goal was to provide full-time jobs with benefits, medical, et cetera.
• 1:17
  And this was really based off of the belief that talent distribution equally, but opportunity is not.
• 1:22
  And the right way to really solve for a sustainable lift out of poverty is through work, not aid.
• 1:28
  So the idea was let's do digital upscaling.
• 1:32
  Let's bring people into jobs for the first time.
• 1:35
  to give them the experience to allow them to get a next job and stay in the formal economy.
• 1:41
  So how does that get us to the work that we do now?
• 1:45
  Well, because we have a purposeful hiring model of hiring people on the basis of impact.
• 1:51
  So if we just think about that for a second, especially knowing this is a Wharton podcast, most people hire on the basis of experience or college degrees, et cetera.
• 2:03
  And we were trying to flip the model on its head to say, we want to provide entry into these jobs.
• 2:08
  So we hire on the basis of impact.
• 2:10
  So household income level, you don't need to have a degree because our idea was we could do the digital upscaling.
• 2:16
  And because of that nature of how we do the hiring, we were not expecting people to come in and begin to do software engineering or programming.
• 2:25
  So we started with human judgment.
• 2:27
  And that's really where this started was let's do human judgment tasks, whether it's
• 2:32
  transcribing business cards like back in the day before optical character recognition, things of that nature.
• 2:39
  And I joined almost 10 years ago now.
• 2:42
  And when I joined the company, one of the things I observed is that human judgment is also really necessary in artificial intelligence.
• 2:50
  You need to have humans validate and train the data.
• 2:53
  So we started to shift our human judgment work, which is trainable.
• 2:57
  Basically, you just need to have human judgment and have a platform where you can receive the tasks.
• 3:02
  That's ultimately how we started.
• 3:03
  It was really driven off of our social mission and wanted to create jobs through the digital economy.
• 3:11
  So how do you go from that human judgment work, which presumably doesn't require significant skills and expertise to things that may require more training or that someone
• 3:22
  is just not able to do, just being hired for it?
• 3:27
  That's an excellent point and observation is what we've seen as well is that the kind of AI work that we did a decade ago is very different than the work that is being done now.
• 3:38
  So one of the concepts we had in our social mission is really about the future of work.
• 3:45
  those skills change.
• 3:46
  Our goal is to build digital skills that allow people to stay employed and in the formal economy.
• 3:54
  So what happened is we ended up really needing to
• 3:56
  shift a significant amount of focus into training.
• 3:59
  So not only what is AI and how do you leverage our platform, we also leverage our platform to sort of isolate those things that are human and let the machines do kind of the heavy
• 4:08
  lifting, if you will.
• 4:10
  And we ended up building domain expertise.
• 4:13
  So we have people who've been doing autonomous vehicles and are deep experts in understanding how to do LIDAR and sensor fusion.
• 4:21
  all the way to building partnerships with local universities to support things like domain expertise in GenA
• 4:29
  And can you talk a little bit more about what some of those tasks are that you do now, especially in terms of Gen AMR?
• 4:37
  So we primarily focus on both supervised fine-tuning as well as initial model development.
• 4:44
  So we provide either the initial training data sets.
• 4:48
  We've been focusing quite a bit on multimodal models.
• 4:51
  So the combination of video or images in context with text or passages.
• 5:00
  AI agent type work with HR, legal, finance as an example.
• 5:06
  So those are the types of work that we're doing now.
• 5:10
  And it can also include validating synthetic data.
• 5:12
  So the cost of developing these models is quite high, and they require vast volumes of data.
• 5:17
  whether we are validating synthetic training data, we are helping build some of the training data, or we are fine tuning and doing things like prompt engineering.
• 5:29
  There's all these challenges around accuracy and bias in AI systems.
• 5:34
  Does the work that your workers do address some of those issues?
• 5:41
  Yeah, I'm a big believer and I think a lot of others are that AI models, right, to develop responsible AI is really on four different pillars, right?
• 5:50
  One is related to data provenance.
• 5:51
  So where did the data come from?
• 5:53
  Do you have the right data to create a, you know, fit for purpose application, right?
• 5:59
  Because if it's too little, too much, you know, there's a whole bunch of challenges that can result in bias in the model.
• 6:05
  So we really focus on that data provenance, that data governance.
• 6:08
  But the other couple of pillars are human-centric validation.
• 6:12
  You can leverage models to build models, but you can imagine you want to go from A to B, you might end up somewhere over at C or D.
• 6:19
  There's not a model evaluation or checkpoints in place.
• 6:22
  So we provide that level of human validation.
• 6:25
  But also, we spend a significant amount of our time talking to our clients about what does good look like?
• 6:32
  I know it sounds like a simple question, but really understanding what is your use case?
• 6:36
  how are we expecting to perform really feeds into the process of normally ensuring that you have the right data, but you're doing the right validation checkpoints.
• 6:43
  And you can actually confirm that even after that model is deployed and new data is being ingested, that you know what it looks like, right?
• 6:51
  You can validate and ensure that it stays performant.
• 6:56
  So for someone who's not familiar with how the model development and fine tuning process works, is that essentially a worker gets a series of outputs and is saying, this is right,
• 7:07
  this is wrong, and it's going to the reinforcement learning engine, or is there more to it?
• 7:12
  Yeah, that is one example use case to where you can get a set of data.
• 7:17
  So for example, in prompt engineer or fine tuning, the model has responded to a prompt and the human is saying, okay, this is accurate.
• 7:28
  It's not accurate.
• 7:30
  The sentiment is off.
• 7:31
  That's super awkward wording, you know, as an example.
• 7:33
  And so that's what a human is doing is just doing some of that fine tuning and saying, hey, the model didn't get it right.
• 7:39
  And then typically the
• 7:42
  the ML engineers who are kind of doing the turning the dials and knobs, right?
• 7:45
  They're doing the weights and bias, the fine tuning.
• 7:48
  We'll then say, okay, great.
• 7:49
  We need to trade more in this aspect to get the model smarter for whatever it might be.
• 7:55
  It might be math or programming, et cetera.
• 7:58
  So that is oftentimes what we're doing.
• 8:00
  In some cases, we're actually validating synthetic data that's used to train the engine.
• 8:04
  We could be validating the, or creating the data.
• 8:09
  So,
• 8:10
  In some cases for specific applications, you need a specific type of data that's not easily accessible through your existing dataset.
• 8:16
  And so that actually needs to be created to initiate the training.
• 8:20
  So it's kind of like a full life cycle.
• 8:23
  Typically it starts very iteratively and I just think of it as a big circle, right?
• 8:27
  So you start, you train the model, you figure out, it, you can it do certain functions?
• 8:32
  You evaluate its, you know, scores and then tweak it.
• 8:38
  And you continue to go through this process.
• 8:40
  sort of over and over again until you get to a point of saying, this model's ready for prime time.
• 8:48
  I would assume in some cases there are cultural or other kinds of issues that become relevant and what is an accurate response for a model, especially if we're talking about
• 8:57
  things like toxicity or something like that, is going to depend on some contingent judgment.
• 9:03
  So if you've got people that are coming from all around the world, how do you ensure that you're matching up the decisions they're making with what the model developer wants?
• 9:14
  Yeah, that's a really, really, I think important point because what does the role of cultural context play?
• 9:20
  given that these models, they know no boundaries.
• 9:24
  I mean, I think that's the thing that's kind of crazy about Gen.
• 9:26
  is that it's not like you're saying, hey, I'm building a, know, a autonomous vehicle that will operate in London, you know, versus around the globe.
• 9:35
  You're talking about literally any topic and any subject that these models, you know, can be asked so that cultural context can become important.
• 9:42
  Like what does the...
• 9:43
  We can see that performance, example, in India, know, with chat GPT is going to look different than does in the, U S in terms of performance.
• 9:51
  So one of the things that we, we do is, is we, try to isolate and understand what is that level of context.
• 10:00
  Now we found that, you know, with the strong English languages that we have in East Africa that we've had, you know, good, we we've been able to, balance some level of training
• 10:11
  with,
• 10:12
  you kind of existing knowledge.
• 10:13
  I mean, the reality is, that, there'll be differences.
• 10:16
  I remember working on a very large e-commerce project.
• 10:22
  You had to identify pudding.
• 10:26
  Well, pudding may be one thing to us.
• 10:29
  What does blood pudding look like?
• 10:31
  What your pudding look like?
• 10:33
  And so I remember...
• 10:35
  in the UK, right?
• 10:37
  As opposed to like a particular thing here, right?
• 10:40
  Exactly, exactly.
• 10:41
  Or a jello pudding cup, right?
• 10:43
  And so those are some of the examples to where that cultural context can become helpful.
• 10:47
  And there are ways to train and sort of guardrail for and do sampling and things of that nature.
• 10:51
  But there is an aspect of that, which is also, I think, one of the reasons why it's very important that you get, you know, diversity, not just in the data, but also the people
• 11:02
  that are actually touching the data.
• 11:05
  And with regard to issues of bias, the challenge is often the humans.
• 11:10
  It's not that the AI system generates these bias results.
• 11:14
  It's training on data and decisions that are made by humans.
• 11:19
  So if you are putting more humans in the loop in the process for developing and improving these models, how do you combat the potential for those humans to add to the biases?
• 11:32
  Yeah, the potential inconsistencies, you know, I think that that's absolutely a factor, but I would probably come bring in, I would answer that in really a couple of ways.
• 11:41
  One is that the model valuation.
• 11:44
  So how do you define quality is a really critical thing.
• 11:47
  Cause if you don't know what you're looking for, nobody ever gets it right the first time, right?
• 11:52
  You may not, you may be starting, you know, the old adage of garbage and garbage out, you may be starting with the wrong starting point, which again, if you don't have all the
• 12:00
  right,
• 12:01
  data, then you get inference bias, right?
• 12:04
  That just occurs naturally because you're missing something.
• 12:08
  The other component of having an evaluation rubric is that you might have underfit or overfit your model, right?
• 12:14
  So if you underfit it means the next time this new data comes in, it's like, whoa, I don't know how to deal with this.
• 12:20
  So it starts with understanding what are we really trying to tweak and work for?
• 12:25
  And then there are a lot of different methods in which you can drive consistency.
• 12:29
  so consistency can be anything from we are leveraging, it sounds ironic, automation basically to make sure that the human loop responses are consistent.
• 12:41
  And it could be everything from automated quality checks, grammatical checks, that you need to ensure consistency, all the way to other approaches, which can include consensus
• 12:53
  building.
• 12:54
  as an example, that creates bias in a crowd situation because in the crowd, you only get paid when you get it right.
• 13:02
  So there are so many numerous examples and studies that say you just keep hitting that button, know, just keep hitting until you get it right.
• 13:10
  And if that's how people are treating that, you know, just to get to the point of getting paid or, you know, I'm really in the Philippines, but I say I'm from the U.S., you know,
• 13:19
  to get to, there's lots of gamemanship that can happen in that place.
• 13:22
  So,
• 13:25
  Even in that scenario, sometimes the volumes in the consensus can also have some flaws.
• 13:30
  What do do to combat those problems?
• 13:34
  From our perspective, we do a couple of things.
• 13:36
  So one is we really focus very heavily on identifying the model evaluation standards and the quality rubric.
• 13:42
  So working with our clients to understand, hey, what is it that we are really validating for and identifying quality and parameters associated with that and that accuracy?
• 13:52
  And then we leverage everything from systematic quality checks to actual quality and quality sampling.
• 14:01
  you know, statistical sampling to ensure that we are seeing a level of consistency.
• 14:05
  And we've got a way to judge what good looks like.
• 14:08
  And in the cases of stuff that's subjective, I mean, it's not, it's not perfect, but it's going to get you to a place of understanding what are those ultimate sort of business
• 14:17
  rules that are necessary to create an accurate output, which could be factual accuracy, which could be grammatical accuracy, which could be tone, sentiment, et cetera.
• 14:27
  You name it.
• 14:29
  And you've talked about the different stages all the way from the original data provenance through the process where these kinds of mechanisms come into play.
• 14:40
  mean, do you have a sense, is there an evolution in terms of where there is the greatest need for these quality improvements, or is it always across the board at every stage?
• 14:53
  Well, I think that's the tricky part about AI is that it doesn't sort of sit on a shelf.
• 14:59
  It grows and changes as the world under our, you know, the ground under our feet changes.
• 15:03
  And so it's an ongoing thing.
• 15:05
  You have to have some level of performance and feedback mechanisms.
• 15:10
  So for models that are put out in production, and we talk a lot about gendered VA, but you could say the same thing for a self-driving car or autonomous vehicles.
• 15:18
  It's the same thing.
• 15:20
  where it may go on a new road and you have to have a feedback mechanism for a user, right?
• 15:25
  Or internally to be able to flag and say, hey, that's not accurate, right?
• 15:29
  Or I don't know if you drive a Tesla or have any of the driven those vehicles before, if it suddenly slams on its brakes, you hit the report button, right?
• 15:39
  To capture the camera information to say, hey, you need to fix this.
• 15:43
  So that feedback mechanism is constant.
• 15:46
  A lot of estimates are that
• 15:48
  It even takes 15 plus percent or more to just maintain the models that are put out there.
• 15:52
  So it's a long way to say that, yes, it does occur at each stage, the sort of level of effort and the costs associated with it reduces ultimately over time.
• 16:06
  And you're more in an evaluation and tuning stage.
• 16:11
  But actually that addressed the next question I was going to ask you, is whether in time there's enough data and enough accuracy in the models that the need for the kind of human
• 16:20
  feedback you're doing decreases.
• 16:23
  It sounds like that still continues because there's always something new.
• 16:28
  I think it does still continue.
• 16:30
  Does the volume continue at the same pace it used to?
• 16:33
  No, I don't think so.
• 16:34
  I mean, we've even seen that in the world today where, you if more can be automated, it will be automated.
• 16:40
  But it's very difficult to, and I would say I think most experts would very, very much agree that you don't want to lose the notion of human centric validation, that something
• 16:51
  is a model, there's not enough transparency that says this is both how it's been built.
• 16:56
  and how it's being checked.
• 16:58
  So kind of when we come back to responsible AI frameworks, right, we talked about data provenance, we talked about, you know, transparency, we talked about model evaluation,
• 17:08
  etc.
• 17:08
  I think those things are really key because at the end of the day, there's going to be, you know, there's some activity that's happening towards regulation and compliance, if you
• 17:19
  will.
• 17:20
  At the end of the day, like really where we are able to balance innovation is going to be in standards.
• 17:24
  right, and open source and, you know, getting everybody to come to the table say, hey, this is the best practice way in which you develop models.
• 17:31
  Because at the end of the day, the adoption is about the user's right?
• 17:36
  So if you don't trust that system, and I think this is kind of this really interesting state we're at, Kevin, is that, you know, I'm in California and there was a bill that
• 17:46
  failed pretty spectacularly, SB 4057, I think I got the numbers right on that, but...
• 17:53
  It had input from Anthropic, had input from Meta, it input from all these different companies.
• 17:57
  So it was the right kind of intention of public-private partnership, but also was pretty flawed.
• 18:03
  So lots of good intentions, ultimately failed because some of those weren't, I think, very pragmatic.
• 18:09
  And there was a real concern of stifling innovation.
• 18:11
  Yet at the same time, every poll that you read on that said eight out of 10 Californians think it's a good idea.
• 18:17
  We want to know how those systems are being built.
• 18:19
  We want to know that if we're
• 18:22
  You're building technology that's going to touch a financial system and say lending services that is not totally biased against a certain type of, or could be subject to
• 18:32
  cyber attack.
• 18:35
  So beyond the specific work that you do at Sama, how do you think we get to a more trustworthy AI environment?
• 18:44
  I think a lot of it is about education and standards.
• 18:47
  So I think it's got to be a combined path of some level, some position.
• 18:52
  I I don't think you can solve innovation through government policies.
• 18:56
  I mean, not to ding, you know, on that, but I think there's a role for a government to play, which is to encourage the right kind of behavior or discourage the wrong kind of
• 19:03
  behavior.
• 19:03
  So I think there is some framework.
• 19:05
  It's a really difficult problem to solve, but I think there's value to identifying higher resistance.
• 19:11
  I think there's value to saying,
• 19:13
  If this technology can be used to can be hacked or otherwise, right, and is embedded into critical areas of like national security, these are important things that we should be
• 19:22
  doing.
• 19:22
  Absolutely to have the right guardrails in place.
• 19:25
  But at the end of the day, the decisions that are being made on these AI models aren't decisions that are being made in Washington, DC or in Sacramento and California.
• 19:33
  It's the engineer who's building it, who has to have that understanding and framework.
• 19:39
  So I'm more of a fan of
• 19:41
  I should say, I think it's a two-pronged approach, is this notion of responsible and transparent AI, getting those guidelines and best practices out there, enterprises who are
• 19:50
  investing heavily in this should take responsibility to educate, train, share, and then that same thing needs to go on the consumer side.
• 19:58
  I a lot of people go and they start entering in like, you know, I know enough to know that if I'm going to leverage any LLM, I'm not going to give or ask it information about myself
• 20:09
  because
• 20:10
  That is me then giving up my own personal data to train that model.
• 20:15
  So I just opt out of the LinkedIn auto AI algorithms to collect my data and send more personalized.
• 20:26
  So that education also exists on the consumer side because the way I look at it even it's very simple.
• 20:33
  Like even hairdryers have warning labels.
• 20:36
  Don't plug it in next to water.
• 20:38
  you need to that kind of visibility, it's multi-pronged.
• 20:44
  Let me jump back to Sama and your business.
• 20:47
  How do you find quality people and then how do you effectively train them?
• 20:55
  So we have a couple of approaches.
• 20:57
  So we have our impact focus.
• 21:00
  So that is the hiring and the basis of impact.
• 21:04
  We go through kind of an AI 101, is everything from just post-digital literacy to what is AI?
• 21:14
  Like, what are the inputs that you're providing?
• 21:17
  How does it train the model?
• 21:18
  And being transparent about that, being transparent about how AI is used in our own platforms.
• 21:24
  and teaching about use cases and sort of quality and business rules associated with creating, you know, predictable and accurate data.
• 21:33
  So we go through sort of a baseline training and then of course, customer specific training.
• 21:38
  And what we are certainly finding with Gen.AI is that oftentimes because of the also somewhat subjective nature of the task that you have to start with some level of domain
• 21:49
  experience.
• 21:50
  So it's gonna be pretty difficult to validate
• 21:53
  a set of Python code or create training data for Python code if you have never programmed before.
• 22:00
  And so this is where we have been focused on our same communities where job creation is quite important, but we're partnering with the University of Nairobi and Swarthmore and
• 22:13
  Makary in Uganda as an example.
• 22:17
  And there's been a longstanding, sorry?
• 22:21
  yeah.
• 22:23
  to try to create an employment ecosystem.
• 22:26
  Sorry, there is a long standing debate about outsourcing in many industries ranging from IT to apparel and the question about whether this is actually benefiting people when their
• 22:40
  railing wage is going to be significantly less than in the developed world.
• 22:44
  So is there anything different about how that plays out within the realm that you're dealing with?
• 22:54
  I think there's a couple of components.
• 22:56
  The first and most important one is, there's a lot of wealth and opportunity that's being created from AI and from the digital economy.
• 23:07
  Why should not the global world participate or should it be relegated to a small group of people in the Silicon Valley or in Austin or in other locations?
• 23:16
  This is a huge opportunity and I think the world is flat.
• 23:22
  There's an opportunity for
• 23:24
  economic prosperity and growth across the board.
• 23:27
  So that's one thing.
• 23:28
  The second component is that things don't cost the same everywhere.
• 23:32
  There's a purchasing power component associated with this.
• 23:35
  it's, you know, you look to any global company and you tell me is the, the, you know, front desk reception person, you're to make the same thing in Bangalore, India, that they
• 23:49
  would in San Francisco, California, that they would in any of those other locations.
• 23:53
  The way that we do this, I'm certainly a big proponent of this, and this is part of both responsible and ethical AI, is transparency in wages.
• 24:02
  So it's very easy for this to be erased at the bottom.
• 24:06
  So certainly what we advocate for in the model that we take on at Sama is we leverage the Anker methodology.
• 24:14
  So that's a globally recognized methodology for identifying living wages, which is based off of these standards to have safe housing.
• 24:22
  to have 10 % savings to afford education, medical care, healthy foods, things of that nature.
• 24:27
  In my location, what is that living wage?
• 24:30
  That is really the representation or tailored approach.
• 24:33
  And so we do that across every location that we work in.
• 24:36
  Say this is what the living wage is, and it typically corresponds, it obviously rises with inflation.
• 24:43
  And as an example, in Kenya, the living wage is going to be about
• 24:52
  almost four times more than the minimum wage, the bare minimum wage.
• 24:56
  Right.
• 24:57
  So, so there is a relative kind of view if you can think about it like that.
• 25:00
  And that's what we look at is we say, Hey, what does the living wage is our benchmark.
• 25:05
  evaluate that every year through a series of surveys, leveraging this methodology.
• 25:09
  And yeah, that, that is both it's, it's, it's fair.
• 25:13
  We, know that, that, that, we're providing to meet the standards and, know, it's not easy living wage goes up more than.
• 25:21
  you know, necessarily market wages do, but that's a commitment we've made.
• 25:25
  And I think the transparency of this notion of full-time employment and living wages is really important because otherwise it is a race to the bottom, right?
• 25:35
  You know, the top, it's just going to continue to get cracked down further and further and further.
• 25:39
  And I think the thing that we should be incentivizing, you know, well, one for humanity, but second for also accuracy is do you want to have the, you know,
• 25:50
  the people that are training your models or working for this, be incentivized to do as much as they humanly can in the shortest amount of time possible.
• 25:59
  You want it to be quality.
• 26:01
  It's really difficult to do that if you haven't set basically a safe and effective work environment.
• 26:11
  And that's the benefit of a formal job.
• 26:13
  You get sick, you still get paid.
• 26:15
  You have to go paternity leave or maternity leave.
• 26:18
  you still have those protections and I think those actually ultimately equate to better work at the end of the day.
• 26:26
  How else do you ensure that you're operating in ethical ways?
• 26:29
  You talked about the original instantiation of the company was as a nonprofit, but now you've raised back your capital, you scale it up a lot.
• 26:37
  What does it mean to realize that commitment to ethical sourcing?
• 26:42
  So we do in a couple ways.
• 26:44
  The one primary way we do it is we hold to our impact hiring practices.
• 26:49
  The goal is to be able to provide opportunities to people who've got the greatest barriers to employment.
• 26:53
  So we do that.
• 26:54
  We track that.
• 26:55
  We have our commitment to living wages.
• 26:57
  So we have an impact team that does that evaluation review every year.
• 27:02
  We report on that in our UNSDGs.
• 27:05
  So we hold ourselves accountable to ensuring that we drive that forward.
• 27:09
  And then we work towards, you
• 27:12
  There are five UN SDGs that we are targeted on specifically, includes a fair and safe working environment, includes gender diversity.
• 27:22
  So one of the things that was part of our core social mission was that we would hire at least 50 % women or people identify as female.
• 27:31
  And that was a purposeful approach we took from a mission standpoint.
• 27:35
  because there's a lot of data that tells you that when women succeed, communities succeed.
• 27:39
  And so we've continued to keep that focus of diversity as well from kind of the bottom up to the top.
• 27:49
  You had an incident with OpenAI where, as I understand it, workers were having psychological consequences of viewing toxic content for the processes they were involved
• 27:59
  in.
• 27:59
  As I understand you, you canceled that contract.
• 28:01
  So I'm interested to hear how you think about making those decisions in terms of what kinds of work to engage in and when to decide that you need to step away.
• 28:10
  Yeah, I appreciate that.
• 28:13
  There are a couple of key things.
• 28:14
  So first off from an OpenAI standpoint, that was a short pilot project to do training.
• 28:19
  It was one of two projects that we've done in harmful content.
• 28:23
  And ultimately the learning from that was that's not part of our core.
• 28:28
  We're not going to do this.
• 28:30
  So we had, it represents two out of probably 3000 engagements and projects we've run over the course of the decade.
• 28:39
  So it was
• 28:40
  learning to focus on our core competencies.
• 28:43
  And we did a couple of things that were really, I think were very good sort of learnings.
• 28:49
  The first thing we did is we said, okay, how did we get here?
• 28:55
  We didn't have a clear enough view of what we will do and what we will not do.
• 28:59
  So we instituted a service line boundary.
• 29:02
  And that's basically something we instantiated up through both our board and into our contracts.
• 29:08
  One example is we will not
• 29:10
  do any projects that involve harmful content.
• 29:13
  We also took that step further and said, okay, are there industries or areas that we think we should just clearly draw that boundary on?
• 29:19
  And we made a call to say, we're not gonna focus on weapons.
• 29:23
  We are not gonna focus on, you know, big tobacco.
• 29:25
  We just say, hey, these are things that we're gonna just draw on and say, hey, let's stay clear.
• 29:30
  But the thing that is very interesting is that there is, the world changes, right?
• 29:34
  What is harmful now could be different.
• 29:37
  in 20 or 30 years, think about, know, urban safety, could that data be taken to, you know, ultimately become, I don't know, invasive or surveillance?
• 29:49
  Like there's just, there's sort of all sorts of things.
• 29:50
  said, Hey, we don't know everything now, but let's create these boundaries.
• 29:54
  And then let's also create something that we call an ethics guild, which is representative of our associates in the Stafford hub.
• 30:01
  So the people are doing the annotation, no management, you know, people from our research team, marketers, you know, et cetera.
• 30:07
  a cross-functional team that anybody in the company, if we have a new opportunity or client we're pursuing, client we're pursuing can raise and say, this seems to fall into a
• 30:17
  gray area.
• 30:19
  Like it's, you know, it's clearly it's not harmful or it's not in this industry, but it seems like it's gray.
• 30:24
  And so that process has worked really well.
• 30:26
  We've had maybe a dozen or so cases raised since we launched the Ethics Guild.
• 30:31
  And like one example was,
• 30:35
  This is work that's right in our wheelhouse.
• 30:36
  It was like computer vision, self-driving cars or something, smart cities or something to that effect.
• 30:43
  But somebody said, wow, we've actually done some observation and this company is actually based in a country that has a really, really bad history of violating human rights and
• 30:54
  some other things.
• 30:54
  So it got broad.
• 30:56
  We looked at that and then said, okay, maybe one of the aspects of this is that countries or other
• 31:04
  you know, areas that we say these, you know, companies based in these countries, we would not go down that path.
• 31:09
  Right.
• 31:10
  So that was an example of where something new came up.
• 31:13
  We didn't have the, you know, we had a lot of things documented, but we had a mechanism to capture, you know, something new and make a draw line in the sand.
• 31:21
  So yeah, it's been, you know, there's definitely some learnings from that, from that situation, but I think we have tried to channel that into, you know, just continually
• 31:31
  improving and being really clear.
• 31:35
  And last thing, it is a big world and AI is a technology that has implications everywhere, but the vast majority of the major AI development labs are in a small number of countries.
• 31:49
  What can be done to better globalize these systems so that they are most useful to people in other places like the development world?
• 31:59
  That is such a good question.
• 32:02
  It's one that I do deeply think you need greater diversity, right?
• 32:06
  I AI knows no boundaries, right?
• 32:08
  It's not like we built something here.
• 32:09
  It's not relevant there.
• 32:10
  I think one of the ways we can do this is we can create ecosystems.
• 32:15
  So I was just in a conversation with some folks from the G7 who were really talking about, how do we find this balance of government support?
• 32:24
  You know, some people are like, well, I think it's about, you know, compliance and, and, you know, regulation and we're going to tell folks how to do this.
• 32:29
  And then somebody was around and they said, you know, isn't that kind of like the opposite?
• 32:33
  Like what if you're in a developing nation, like say some sub-Saharan nations in sub-Saharan Africa, wouldn't we actually want to instead focus on creating an ecosystem?
• 32:43
  I was like, yeah, that's, that's at the end of the day.
• 32:46
  What's really key is that the way that we get greater diversity and greater representation is we get the, you know,
• 32:54
  where the governments can engage is build STEM programs, you know, like focus on providing, you know, labs and GPUs and build those ecosystems that ultimately there's
• 33:05
  great adversity in both the talent and the technology that's being developed.
• 33:10
  So I think that's really the right broader way to do it.
• 33:13
  And then in the meantime, I think that there is a great opportunity for, you know, the companies, if they're thinking about this in that kind of, you know, broader responsible
• 33:21
  AI framework to say, hey,
• 33:23
  you know, who can I involve both in terms of the data and through the supply chain to create something that's more representative?
• 33:29
  So that's probably how I'd answer that.
• 33:32
  Great, Wendy, thank you so much for your time.
• 33:35
  Thank you, Kevin.