Latest / The Road to Accountable AI / Suresh Venkatasubramanian: Blueprints and Redesign: Academia to the White House and Back

Transcript

Suresh Venkatasubramanian: Blueprints and Redesign: Academia to the White House and Back · The Road to Accountable AI

• 0:01
  Suresh, welcome to the Road to Accountable AI.
• 0:04
  Thanks for having me, Kevin.
• 0:06
  You've worked for a number of years as a computer scientist, among other things, on
• 0:11
  issues of algorithmic fairness.
• 0:14
  So how far along are we in terms of technical capabilities?
• 0:19
  If a company uses best practices today in terms of devising training data and models
• 0:26
  and outputs, should they be reasonably confident that problems about bias can be
• 0:31
  addressed?
• 0:33
  I think most companies today could be reasonably confident that with the right
• 0:37
  people in place and the right teams, there's a whole suite of tools, ideas,
• 0:42
  concepts they can rely on to do a pretty good job in putting out products that they
• 0:47
  can be proud of that are indeed mitigating various forms of bias.
• 0:53
  I think we're not, I would say, 100 % of the way there yet.
• 0:58
  Where are the biggest gaps?
• 1:01
  I think the gap is such as really in...
• 1:05
  in going from what I'll call more sort of bespoke boutique sort of specific things
• 1:13
  that have been known to work in different cases to a general set of prescriptions
• 1:20
  that is both general enough to cover all the different issues that might come up,
• 1:25
  but specific enough that can be tailored to any organization's own needs.
• 1:29
  And I think that's where I think I just came back from a workshop where we were
• 1:33
  talking about, you know, this idea of
• 1:35
  bridging predictions and interventions.
• 1:37
  And this issue of where the rubber meets the road and how you actually
• 1:44
  operationalize these ideas that are out there is I think where the last mile sort
• 1:50
  of problem is, I guess.
• 1:53
  What do you think that will get there?
• 1:55
  Is it a research problem or is it more about translating what computer scientists
• 2:00
  are doing to actually be useful for organizations?
• 2:05
  I don't think it's a translation.
• 2:06
  I think of translation as I've said the correct thing and now I need to translate
• 2:09
  into your language.
• 2:10
  I think it's more of a conversation.
• 2:12
  I think we are having those conversations in various places, but the conversation
• 2:17
  has to be something like, okay, here's my organization with my specific concerns, or
• 2:21
  rather here's my organization and I've heard about these issues, I'm worried
• 2:25
  about them.
• 2:25
  And then someone in their organization can say, all right, let's break it down.
• 2:30
  What are your concerns?
• 2:31
  Where are the things you want to prioritize?
• 2:34
  Okay, now here's a bunch of tools we can bring in.
• 2:36
  Let's deploy them in, let's build a governance structure on them.
• 2:39
  Let's see what results are getting, let's adapt.
• 2:42
  So that conversation, we have the pieces for each of it, but I find myself
• 2:46
  constantly saying, okay, this is just what I just told you right now.
• 2:49
  This is what you need to do.
• 2:51
  You need to build a team, you need to think about values, you need to think
• 2:53
  about implementations, you need to think about monitoring and so on.
• 2:56
  So I think that's, so there is research that is going to be teed up from those
• 3:00
  conversations.
• 3:01
  And I...
• 3:01
  As an academic, I want to hear more about this conversation so that we know what
• 3:05
  questions we should be asking and trying to answer.
• 3:08
  But those conversations are key.
• 3:09
  And having those conversations in a way that some of it can be surfaced in the
• 3:12
  public domain is also key.
• 3:15
  How much has the rise of generative AI changed the way that you would answer
• 3:20
  those questions?
• 3:22
  At one level, it has not at all.
• 3:24
  Another level, it has confused everything.
• 3:26
  And I think it has confused everything at say the C -suite level, where suddenly
• 3:30
  executives are very panicked about generative AI and want to make sure
• 3:32
  they're ahead of the curve or they're not being left behind.
• 3:35
  So there's a lot of push within organizations.
• 3:37
  So we need to, you know, we need to gen AI our whole company.
• 3:40
  So put it in everywhere, right?
• 3:41
  And I think the problem is that no one really knows what that means.
• 3:44
  And I think I've been hearing from people that once they actually sort of get down
• 3:47
  to it, roll the sleeves and say, okay, what does it mean to bring generative AI
• 3:50
  into our organization?
• 3:51
  They realize a whole bunch of risks, a whole bunch of concerns, a whole bunch of
• 3:54
  vulnerabilities that they can get exposed to if they do that.
• 3:56
  And so then they have to say, well, wait a second.
• 3:58
  I have to think about this.
• 3:59
  In the meantime, though, all the other places where organizations have been using
• 4:04
  AI or what I'll call predictive AI as opposed to generative AI, those issues
• 4:08
  still remain and we still need to be concerned with them.
• 4:10
  But because all of these get lumped together under the rubric of AI, the
• 4:15
  confusion and the panic about generative AI is causing...
• 4:19
  unreasonable confusion and panic about predictive AI where we know a ton more.
• 4:23
  And I think that's something where I find myself often having to say there's this
• 4:27
  kind of AI tools as that kind, you're using a lot of that kind and we can do a
• 4:32
  lot there.
• 4:33
  You're getting panicked about this kind, that's a whole different discussion we
• 4:35
  should have.
• 4:37
  Yeah, so I mentioned in that question about, you said we know a lot more about
• 4:41
  the predictive models.
• 4:43
  So if I am deploying a supervised machine learning system, I've got data scientists
• 4:50
  who can use various algorithms and tools to have some sort of fairness function be
• 4:56
  put on the outputs.
• 4:57
  What's different if I'm using a large language model or a foundation model like
• 5:02
  a GPT?
• 5:04
  Fundamentally, if you're using a large language model or some other foundation
• 5:07
  model, unless you've built the model yourself, which is highly unlikely, you're
• 5:13
  using a model that's come from somewhere else.
• 5:15
  Either it's an open source model that you brought into your system, or it's an API
• 5:19
  -based access to, you know, chat GPD or GPD 4 or 4 .0 or something like that.
• 5:24
  You no longer have full...
• 5:26
  If you haven't built in -house, which for the vast majority of people is true, you
• 5:30
  no longer have control over the system that you're actually working with, which
• 5:33
  in...
• 5:34
  In contrast to say a supervised learning system that you build in -house, you may
• 5:39
  not understand all the details of why it's producing outputs, but you have full
• 5:42
  control over the data, over the training, over the model.
• 5:44
  So that's one problem.
• 5:46
  Secondly, we don't have really good tools to do any kind of mitigation or
• 5:52
  intervention in generative AI systems, which means if I try to evaluate the
• 5:59
  behavior of such a system and I want to check for various kinds of biases,
• 6:02
  It's a lot harder to do that because the space of possibilities where it can
• 6:08
  perform a certain way is much larger because it's really about natural language
• 6:11
  prompts.
• 6:12
  Unless I have a very controlled set of prompts and I only allow those access to
• 6:15
  it and I test that rigorously, it's really hard to understand what the system is
• 6:19
  doing.
• 6:19
  And if I discover some problems, it is really hard to figure out what to do about
• 6:23
  it at that point.
• 6:24
  So most of the framework and the tools we've built up that work well with
• 6:28
  predictive systems, whether it's...
• 6:30
  fairness mitigations, whether it's explanations, whether it's bridging this
• 6:34
  gap between prediction interventions, those don't work anywhere near as well, or
• 6:40
  at all in some cases, with foundation models.
• 6:43
  And we're in a position now where we have to rebuild our understanding of those
• 6:47
  systems.
• 6:49
  So you spent two years, I believe, at the White House in the Office of Science and
• 6:53
  Technology Policy working on AI issues.
• 6:56
  Let me ask you first, generally, what was your experience as an academic coming into
• 7:01
  that environment?
• 7:03
  Well, you say two years, it was actually one and a quarter, but it felt like 10.
• 7:06
  So who knows what that means.
• 7:08
  It was quite a different experience.
• 7:11
  I mean, I'd done a little bit of, I guess, dipping my toe into the lake of policy
• 7:16
  work at the state and local level.
• 7:18
  So I had a teeny bit of exposure, but it was nothing like what had actually
• 7:23
  happened when I showed up in DC at the White House.
• 7:27
  There are many ways in which policy work is like academic work, and there are many
• 7:31
  ways in which it's
• 7:32
  totally different.
• 7:34
  And I had to figure out which of my skills I could transfer over and which I had to
• 7:38
  rebuild and just come up and learn brand new things.
• 7:40
  So in one way, it's great.
• 7:41
  You're a professor for a while.
• 7:43
  You think, OK, every day is going to look the same as every next day.
• 7:46
  And then suddenly, you get thrown into an environment where you feel like a first
• 7:49
  year.
• 7:49
  I kept telling people I felt like a first year grad student again.
• 7:51
  And that was kind of fun and intense.
• 7:54
  But yeah, you learn a lot very quickly.
• 7:57
  Otherwise, if you want to survive, you have to.
• 8:01
  Yeah, I mean, one of the things that I find curious is your take that people
• 8:04
  don't necessarily understand about the US and the White House is obviously the White
• 8:07
  House is this incredibly powerful institution.
• 8:11
  And there are many people who are working for different parts of the White House or
• 8:15
  the executive branch.
• 8:16
  But relative to the incredible breadth of issues that come before it, it's actually
• 8:23
  really tiny.
• 8:24
  The number of people that are engaged in any particular area.
• 8:28
  is pretty small.
• 8:29
  So I'm curious what you felt like being part of OSTP engaging on these AI issues
• 8:37
  and how much the people that you were with were able to actually move the needle.
• 8:43
  Yeah, I mean, it's very true.
• 8:45
  The White House is very small.
• 8:46
  I think OSTP in its role as a convener of science and tech policy work across the
• 8:50
  federal government had a much broader view of who was doing what where.
• 8:54
  So in that sense, I was, you know, dealing very quickly with the full complexity of
• 9:00
  the entire US government, or at least those people thinking about AI, which in
• 9:03
  every agency was so many people who were thinking about this and trying to do a
• 9:06
  good job of it.
• 9:07
  So, so I got a much bigger view, but you're right that within the White House
• 9:10
  itself in terms of change making and
• 9:12
  advocating for change, it's a small group of people you have to work with and
• 9:16
  understand the network structure within.
• 9:20
  So that takes some getting used to and just understanding all the relationships
• 9:23
  that we built up, all the key players.
• 9:26
  And often titles tell you a little bit, but don't tell you a lot about who's an
• 9:30
  important actor in the space and whose ear you need to bend, you know, for some
• 9:34
  issues.
• 9:35
  So that takes some learning to do.
• 9:36
  But yeah, the breadth of issues being covered is fast, which means that
• 9:42
  Again, and this is different from being an academic where you, as an academic you
• 9:45
  specialize in some area and you become very, very good in a special area.
• 9:48
  When you're working in government and you're working at that level, you have to
• 9:51
  be a generalist.
• 9:52
  So you have to be able to suppress that constant feeling of, I don't quite know
• 9:57
  exactly what I'm doing here.
• 9:59
  I don't know, I'm not the expert on this and someone else must be.
• 10:02
  And do the best you can.
• 10:03
  Reach out to expertise when it's appropriate.
• 10:05
  Sometimes you can reach out to outside experts.
• 10:07
  Sometimes you can't based on the rules around what you're doing.
• 10:10
  But you have to do the best you can.
• 10:11
  I think a lot of policy work is engineering that way.
• 10:13
  You do the best you can with what you have right now at the moment.
• 10:17
  You can't wait for this perfect solution to appear.
• 10:20
  That's not a luxury you have in the policy world.
• 10:25
  What was the function or vision for the blueprint for an AI Bill of Rights, which
• 10:30
  was really one of the central things that you were involved with developing?
• 10:34
  The premise, I think, was that coming into the Biden administration, it was becoming
• 10:42
  very clear that questions on AI, especially questions on AI in society,
• 10:48
  were going to be central to the administration's thinking about race and
• 10:52
  equity within the government, about how to think about tech policy more broadly as
• 10:56
  these questions were surfacing.
• 10:58
  But there wasn't a very clear articulation of
• 11:04
  What is it we should be caring about?
• 11:06
  There were different ways to think about it.
• 11:08
  I mean, the EU had started their process by then already, and they were well into
• 11:11
  their process for the AI Act.
• 11:13
  And different groups were putting out their own arguments about what to be
• 11:17
  thinking about.
• 11:17
  A lot of that was very centered around the technology itself.
• 11:20
  A lot of us said, OK, here's a piece of tech.
• 11:21
  Here's what you should do.
• 11:23
  I think the vision inside OSTP that folks were thinking about was, what does it mean
• 11:29
  to center people's rights and impacts on people?
• 11:34
  first and think about the technology second.
• 11:38
  And that, I think, was in many ways a shift from how the discourse was happening
• 11:43
  around tech, but a shift that needed to happen.
• 11:46
  And so the overall effort to generate the blueprint was to say, if government isn't
• 11:52
  the job of understanding what people need, understanding and protecting people, even
• 11:57
  from themselves, and what does that look like in an AI -powered society?
• 12:02
  What rights should people have?
• 12:04
  as we barrel towards this algorithm in the future.
• 12:06
  And that was the genesis of the work that led to the blueprint.
• 12:12
  Could you summarize what the blueprint accomplished?
• 12:16
  Well, the blueprint had sort of five things in it, easy to state things that,
• 12:21
  you know, anything, any AI systems or any technology that affects our lives in any
• 12:26
  material way, our rights, our opportunities, our, you know, our access
• 12:30
  to vital services should have, should be safe and effective.
• 12:35
  It should work and it should work for the purpose it was intended.
• 12:38
  They shouldn't discriminate.
• 12:39
  They should use our data very sparingly, if at all.
• 12:43
  They should be transparent and accountable and we should know how they work and why.
• 12:48
  And we should have, you know, what I like to call dial zero for operator, right?
• 12:51
  If you have a backup option, have a way to sort of talk to an actual person.
• 12:55
  So you're not constantly relying on going back and forth between different piece of
• 12:58
  technology or chatbots or whatever.
• 13:00
  That's it.
• 13:01
  The rest of the document really is about how you go about doing this and why we
• 13:05
  need these in the first place.
• 13:06
  So I think there are two key parts to what makes this document, you know, not three
• 13:09
  pages, but 75 pages.
• 13:11
  which is really making the case for why these principles are important.
• 13:14
  A lot of people are like, well, that all sounds great, but who cares?
• 13:17
  And then, so one of the things that document does is document very clearly
• 13:20
  what happens or what has happened in the past with evidence when one of these
• 13:26
  principles, these rights gets abrogated.
• 13:28
  And so that was a part of the document.
• 13:29
  Another part, and this was very important to us since we were, you know, computer
• 13:32
  scientists sitting in OSTP saying, none of this is a pie in the sky ideal.
• 13:36
  None of this is, you know, crazy stuff that no one can do.
• 13:39
  This is real.
• 13:39
  This is practical.
• 13:40
  this could be done right now, and here's how.
• 13:43
  And so listing out a bunch of different things that you could do and questions you
• 13:46
  need to focus on, some of which would need new research, but some of which could be
• 13:49
  done today.
• 13:50
  And so that was what the document had.
• 13:52
  In terms of impact, it's been almost two years in now.
• 13:56
  And I think I would say that the major impact the document has had, and that's
• 14:00
  been personally pleasantly surprising to me really also, is how it has changed the
• 14:06
  discussions around how we talk about.
• 14:07
  AI in society, right?
• 14:08
  It has done exactly what it set out to do, which is put a marker for how the
• 14:13
  government views the role of the government of regulation of protections
• 14:19
  for people in the age of AI.
• 14:21
  And you can see the DNA of the blueprint in the executive order and the OMB
• 14:26
  guidance and a bunch of state legislation and the number of states that have called
• 14:29
  for their own AI Bill of Rights.
• 14:31
  And that's sort of a pleasing thing to see.
• 14:36
  Is it still relevant and useful after Biden administration executive order, NISQ
• 14:42
  risk management framework, potential legislation and so forth?
• 14:45
  Is there still a value in the blueprint?
• 14:48
  definitely.
• 14:49
  Because I think one thing that has become very clear and that was clear when the
• 14:53
  document was being written was that it was a first step, not a last step.
• 14:57
  Right?
• 14:57
  So it laid out these principles.
• 15:03
  And if you notice when I said technology, I wasn't very specific about AI.
• 15:06
  It applies to all kinds of systems, which means that in some degree, not in every
• 15:10
  degree, but in some degrees, it's future proof.
• 15:12
  It says, you know, when you have new systems that are going to make decisions
• 15:15
  about our lives or assist in decisions of our lives,
• 15:18
  These are the principles you need to have.
• 15:20
  I don't care if it's generative AI.
• 15:21
  I don't care if it's predictive.
• 15:22
  I don't care what it is.
• 15:23
  I think to that extent, as new technology is coming out, as new things are coming
• 15:27
  out, it's very easy to go back to the blueprints, OK, how should we care about
• 15:31
  this?
• 15:31
  It's still there, and it's still valuable.
• 15:33
  So I think it is continuing to be a reference point for organizations, for
• 15:38
  governments, and for students, and for people thinking about AI and society
• 15:42
  broadly.
• 15:44
  And that's very, very powerful.
• 15:49
  What's your view about the executive order that was subsequently issued by the Biden
• 15:53
  administration?
• 15:55
  I think it's quite good.
• 15:56
  I think it covered a lot of different areas.
• 15:57
  It was a very long executive order, I believe almost the longest in history.
• 16:02
  That is what it is, but it also shows that you needed to have something elaborate
• 16:07
  because this area is so complicated and touches so many different parts of
• 16:11
  government.
• 16:12
  So I think the administration, and I would say I had nothing to do with the putting
• 16:17
  out of the executive order or the drafting of it.
• 16:19
  So saying that upfront, I think it did a good job in covering all the different
• 16:22
  areas that people had been concerned about.
• 16:25
  and trying to make progress on them.
• 16:26
  And we've seen since then, you know, the 90 day deadlines, the 180 day deadlines,
• 16:30
  all the things that have been happening.
• 16:31
  We've seen the OMB guidance come out, it's been quite solid in its own right.
• 16:35
  I think there are obviously, again, there are places where it could have done
• 16:39
  better.
• 16:39
  And there are places where we would hope that more would come.
• 16:42
  I think there are big gaps, for example, in thinking around criminal justice and
• 16:46
  thinking about law enforcement more broadly and its intersection with national
• 16:48
  security.
• 16:49
  There's obviously the national security memo that will eventually in some form,
• 16:53
  you know,
• 16:53
  Some of it will be classified, some of it might appear.
• 16:56
  So there's things that can be done.
• 16:57
  But I think that's the point, right?
• 17:00
  It's a constant process of refinement.
• 17:02
  And the blueprint said, OK, here's what we need to do.
• 17:04
  The executive works, like, here's how we need to concretize this.
• 17:07
  The one began says, here's how we need to concretize it even more specifically.
• 17:10
  And I expect this to keep happening.
• 17:12
  I think if we just sit back and say we're done, that'll be the mistake.
• 17:14
  We're not done.
• 17:17
  Yeah, and it frustrates me somewhat and I would guess probably you too when people
• 17:21
  dismiss what's happened in the US and say, well, there's no enforceable comprehensive
• 17:25
  AI law and therefore all of this is just principles.
• 17:29
  But it sounds like what you're saying is that at least there has been a good
• 17:32
  response to the blueprint and understanding that there's a lot more to
• 17:35
  it.
• 17:37
  Yeah, and I think, you know, I mean, Congress is working on what Congress will
• 17:40
  work on.
• 17:40
  States are working on things.
• 17:42
  Colorado just passed, I think, what it would be regarded as the first
• 17:45
  comprehensive AI accountability legislation, which is not bad at all.
• 17:48
  Other states are doing this.
• 17:49
  California is working on one of their own.
• 17:51
  And, you know, so it's happening, I think.
• 17:54
  And that momentum is very important.
• 17:56
  I think people don't realize that we tend to want the shiny thing, the one thing
• 18:01
  that will solve all our problems, you know.
• 18:04
  Government is complicated, society is complicated, you shouldn't expect, in
• 18:07
  fact, the only way for these things to work is to bring everyone along, which
• 18:10
  means bring governments along, bring the private sector along, bring companies
• 18:14
  along who put out their own completely voluntary guidelines that they will
• 18:18
  follow, where people could say, well, no one's forcing you to do it, why are you
• 18:22
  gonna do it?
• 18:22
  But they're gonna do it anyway, because companies wanna make sure that they are
• 18:25
  not doing things where they could get into trouble later or where it could affect
• 18:28
  their brand or affect their image, as you know so well.
• 18:30
  I mean...
• 18:32
  People are going to do this and the trick is to bring everyone on board and bring
• 18:35
  them along for the ride rather than saying that we need the one, you know, shiny
• 18:40
  thing that everyone will follow.
• 18:42
  And that's complicated.
• 18:43
  It looks multi -dimensional.
• 18:45
  It looks tricky.
• 18:46
  It looks like missed.
• 18:47
  It looks like, you know, responsible air practices within, you know, a company.
• 18:51
  It looks like what the IAPP is putting out.
• 18:52
  It looks like all these things together, each one of them is not going to solve all
• 18:57
  the problems, but together they can.
• 19:00
  I mean, what's most important to create that alignment of incentives?
• 19:05
  Because you're right, just the fact that companies don't have legally enforceable
• 19:08
  obligations doesn't mean that many of them don't actually want to and take serious
• 19:13
  action.
• 19:14
  But there are cases where there are gaps when there isn't enforcement.
• 19:17
  So do you have any thoughts about what, from a design standpoint or from what
• 19:22
  comes out of places like the White House that can align those incentives properly?
• 19:28
  I think.
• 19:31
  It's hard to align incentives only with carrots.
• 19:34
  You do need some sticks.
• 19:35
  There's no way around that.
• 19:36
  I know there are a lot of the tech sectors advocating for purely voluntary
• 19:41
  guidelines.
• 19:42
  They will not work.
• 19:44
  Not because companies are, you know, evil and are going to do bad things.
• 19:48
  It's just that, again, ultimately, if you're in the business of running, if
• 19:52
  you're the business of running a business, your goal is to generate profits for your
• 19:55
  shareholders.
• 19:56
  And that is going to take priority.
• 19:58
  And if you don't have an
• 20:00
  a forced incentive to do the right thing, you're going to try to find ways to do as
• 20:05
  little as possible.
• 20:06
  And that's not, it doesn't make you bad, it just makes you a natural creature of
• 20:10
  your own incentive.
• 20:10
  So you do need some sticks.
• 20:12
  So I think the role of government is critical.
• 20:14
  The role of the regulatory agencies is critical.
• 20:17
  But I think that itself won't be enough, right?
• 20:18
  If the only thing stopping us from robbing our neighbors was the police, that would
• 20:23
  not work.
• 20:23
  There aren't enough police to stop us from robbing our neighbors.
• 20:25
  It's also just that we don't want, hope.
• 20:27
  All of us don't want to do that.
• 20:28
  We just don't think it's the right thing to do.
• 20:30
  And I think as an academic teaching students who are going out and working at
• 20:33
  these companies, part of my theory of change is we just need to play the long
• 20:38
  game as well and just change how we think about deployment of AI systems so that in
• 20:45
  a future very close by now, not putting out a system that has safeguards and bias
• 20:52
  mitigation will seem like the dumb thing to do.
• 20:55
  And that anyone putting on a system like, what, are you crazy?
• 20:57
  You didn't put bias mitigation, which is what we do now with other kinds of
• 20:59
  testing.
• 21:00
  If you put out a system that doesn't do some basic unit testing, your software,
• 21:03
  your project management are like, are you nuts?
• 21:05
  I want to have the same are you nuts reaction for guardrails and responsible AI
• 21:09
  practices as well.
• 21:10
  So that just becomes a matter of course.
• 21:12
  And I think we can get there.
• 21:15
  Obviously, the European Union has a somewhat different approach, or at least
• 21:21
  in terms of the order, they have the AI Act that has been adopted.
• 21:25
  It's a very detailed comprehensive prescriptive law.
• 21:29
  So what do you think they got right and what do you think they got wrong?
• 21:32
  I think there's a lot they got right.
• 21:33
  I think they were, you know, you've got to give them credit.
• 21:35
  They were the first ones to pass a law and everyone outside the US and maybe even
• 21:40
  within the US is going to sort of adjust what they're doing now based on the AI
• 21:43
  Act.
• 21:43
  A lot of countries are looking at what the AI Act looks like and they're trying to
• 21:46
  shape their own laws.
• 21:47
  They're going to say, okay, we're going to work with the EU.
• 21:48
  We need to adjust accordingly.
• 21:50
  So there's a lot that's good in there.
• 21:52
  I think they had some struggles with their, you know, scoping the definition of
• 21:56
  AI.
• 21:57
  I think there's a lot more to be seen about how their own...
• 22:00
  government office that they're going to set up to do some of this evaluation
• 22:03
  auditing plays out.
• 22:05
  So I think it's early to say how this will work.
• 22:08
  I think there's some good signs and we just have to wait and see.
• 22:13
  What does it take to, going forward when there is legislation, get academics and
• 22:20
  others from the private sector to engage in working through some of those issues?
• 22:25
  I mean, it's hard, right?
• 22:26
  Because for one thing, you would like academics, for example, to work in
• 22:30
  government, like for people to work at NIST, at the AI Safety Institute, right?
• 22:33
  Congress has put up this ad for people to help with AI safety work.
• 22:38
  And if you're getting a salary of roughly $3 gajillion to go work in Silicon Valley,
• 22:43
  it's hard to go work in the government for much less.
• 22:46
  So that's one problem, I think, at least in the US.
• 22:48
  The economic incentives for undergrads, I mean, I talked to a lot of students, they
• 22:52
  would love to go work in AI for society, but...
• 22:54
  The kind of salaries they see just working in Silicon Valley are very high.
• 22:57
  That's one thing.
• 22:58
  I think just timing, if you're working as an academic in an institution, if you're
• 23:05
  pre -tenure, for example, it's very hard to do this kind of work.
• 23:07
  If you're post -tenure, maybe you can.
• 23:09
  But then the funding sources, figuring all these logistical details can be tricky to
• 23:12
  do.
• 23:13
  I think also it's hard because, as I said, you need to be very specific for an
• 23:19
  organization as to what they need.
• 23:20
  which means almost you need to be embedded within an organization or have very close
• 23:23
  understanding of that organization, which means either that you're working for that
• 23:29
  company or you're consulting with them in some way, but then you also have to work
• 23:33
  with government or talk to people who work in government.
• 23:35
  And so some of these communication pathways are hard to navigate.
• 23:38
  You can't always speak freely about what you want to sort of understand.
• 23:42
  So there are a lot of, I think, challenges to creating the...
• 23:47
  the open lines of communication that are needed to share knowledge about what has
• 23:51
  worked.
• 23:51
  I'd love to sort of say, I'm working at company A doing this.
• 23:55
  Hey, you at company B, what have you been doing?
• 23:57
  Can we share knowledge about this?
• 23:58
  And there was a little bit of that happening.
• 24:00
  Even about a couple of years ago, there were folks in Silicon Valley, all the
• 24:04
  responsible AI teams that were talking to each other about these things.
• 24:07
  But then again, you know, various things happened.
• 24:09
  The tech companies decided to retrench and those teams are dispersed.
• 24:13
  And right now, I obviously don't know how well those teams are connected now.
• 24:17
  But you need those kinds of that knowledge sharing as well.
• 24:21
  So since returning to academia, you started a center at Brown on some of these
• 24:27
  issues.
• 24:28
  And I am curious.
• 24:29
  It's the Center on Technological Responsibility, Reimagination, and
• 24:32
  Redesign.
• 24:34
  So responsibility, I think I understand.
• 24:36
  What's the reimagination and redesign?
• 24:39
  I'm very glad you asked that because that's very important to me, the other
• 24:43
  parts of this.
• 24:43
  So I think we all understand what tech responsibility looks like and putting in
• 24:47
  the right guardrails, the right protections.
• 24:49
  But as a computer scientist, I think that...
• 24:56
  we have an opportunity to not just design protections around systems that are
• 25:03
  already out there, but reimagine and redesign systems that are sensitive to and
• 25:12
  address the needs of all people from the get -go.
• 25:19
  For decades, I think, computer science has viewed itself as a discipline that is sort
• 25:25
  of people -free.
• 25:26
  We build the tools, we throw them out there, we don't worry about what happens
• 25:29
  once we throw them out there.
• 25:30
  We don't even worry about why those tools are being asked of us.
• 25:32
  And I think we can no longer do that.
• 25:35
  It's very clear that, you know, even a simple, let me give you a simple example.
• 25:40
  There's nothing to do with technology.
• 25:42
  So ramps, ADA mandated ramps for people with, you know, in wheelchairs and whoever
• 25:49
  mobility issues.
• 25:51
  You could think of it as a solution that only helps people.
• 25:55
  with disabilities.
• 25:57
  And it's true, it does help people with disabilities.
• 25:59
  But every time you're lugging a large suitcase up a system and you need to take,
• 26:03
  and you can take the elevator because it has to be there in a, in a, in a New York
• 26:07
  subway because it's ADA mandated, everyone is benefiting from this, not just the
• 26:13
  group that ostensibly this was designed for.
• 26:16
  If we could have designed those kinds of systems thinking, okay,
• 26:20
  We are building systems that are going to affect a lot of people.
• 26:23
  People have all kinds of different needs.
• 26:25
  How do we reimagine and redesign our technology from the get -go?
• 26:31
  So we are thinking about access for all and not just access for the most or access
• 26:35
  for some.
• 26:37
  That opens up a whole new set of innovations in computer science, a whole
• 26:42
  new way of designing tools that is a bit more sensitive to what people need.
• 26:46
  I mean, we've had...
• 26:48
  fields like human computer interaction, we've had people centered computing, we've
• 26:52
  had these concepts for a while.
• 26:55
  We need to, I think, go beyond human centered and individual centered to people
• 26:59
  centered and society centered computing to build assistance that actually help us do
• 27:04
  what we want to do as opposed to building systems that are making profits for some
• 27:11
  companies that we sort of shoehorn ourselves into.
• 27:13
  So that re -imagine and redesign.
• 27:16
  I think is a way for computer science to be useful as opposed to fighting and being
• 27:21
  playing on the defensive.
• 27:23
  And I think that's really important.
• 27:24
  I think, you know, we it's been important to identify all the places where we need
• 27:27
  to fix errors in technology, but we have to think positively about the future as
• 27:32
  well.
• 27:33
  We can't just be playing defense.
• 27:35
  And that's, I think the core of what, you know, I would like the center to
• 27:41
  represent, not just.
• 27:43
  issues of responsibility, which are very important, but also a better future.
• 27:49
  What might that actually look like?
• 27:51
  Is it essentially talking to lots of stakeholders in designing systems?
• 27:55
  It must be something more than that.
• 27:56
  So what in practical terms are you thinking about?
• 27:58
  So that's the thing, right?
• 27:59
  I mean, it's hard to give it, because every example looks so different and looks
• 28:05
  so distinct, right?
• 28:06
  So there's only so many things I can talk about right now that we're still doing
• 28:11
  work in progress.
• 28:12
  We haven't got papers out yet to talk about yet, so I don't want to sort of
• 28:16
  indicate vapor where it doesn't exist.
• 28:17
  But I think something as simple, I mean, you mentioned stakeholders, but it is an
• 28:22
  important point, right?
• 28:22
  We did a workshop last year where we were trying to hear from community partners and
• 28:26
  what they want from technology.
• 28:28
  And there was a
• 28:28
  constant refrain of, you know, we want more joy.
• 28:31
  We want to be able to use technology to help us, help us build that.
• 28:36
  So sometimes it involves just not being the lead, but sort of standing back and
• 28:40
  helping others craft their vision of what technology can do for them and helping
• 28:45
  them build it.
• 28:47
  And yes, so it does involve a lot of stakeholder engagement, but it's not
• 28:50
  stakeholder engagement to say, okay, we have this product.
• 28:51
  We want to know what you think of it.
• 28:53
  It's more like, you said you wanted to do this thing.
• 28:56
  How can we help you?
• 28:59
  How can we help you build what you want?
• 29:01
  And how can we help that, you know, scale out?
• 29:03
  I mean, there's, you know, I think of, for example, the work that folks are doing
• 29:07
  building, you know, machine translation and NLP tools for processing the Maori
• 29:11
  language, which was an effort that was done very locally within New Zealand, you
• 29:16
  know, with a local group that wasn't hooked up to OpenAI or hooked up with
• 29:20
  these larger companies, but that was done with stakeholder engagement and has
• 29:24
  produced a tool that's very, very helpful for people who need it in the way that
• 29:27
  they want it.
• 29:29
  And I think we can do more of that, and that will actually help us do better
• 29:32
  computer science as well.
• 29:34
  So I'm still figuring this out exactly what this looks like.
• 29:37
  I think we're working on individual projects with different groups.
• 29:40
  Stay tuned.
• 29:42
  Yeah, well, it would seem like potentially generative AI offers some of that
• 29:48
  opportunity.
• 29:49
  And now it's not just a matter of, I want to build this product or the system.
• 29:53
  Let me get a bunch of data scientists to go and work on it.
• 29:56
  It's I talk to the machine and tell it what I want.
• 30:02
  Yeah, except who built the machine.
• 30:07
  I think you're right.
• 30:08
  I think there is a new sort of platforms being created around the use of generative
• 30:12
  AI.
• 30:14
  I'm not sure OpenAI should get the benefit of all of this though.
• 30:18
  I'm not sure Google should get the benefit of all of this.
• 30:21
  Even for all its openness, I'm not sure Meta should get the benefit of all this.
• 30:25
  So I'm a little worried about that idea.
• 30:27
  I think on the one hand, I see the energy and the excitement and it's really cool to
• 30:31
  play with, you know.
• 30:32
  chatbots and play with the GPD4 and play with GPD itself and sort of see what it
• 30:36
  can do for you.
• 30:37
  It's kind of fun.
• 30:38
  And that fun is important.
• 30:40
  I just want to make sure we're not contributing to the capture of a yet
• 30:44
  another environment by a few companies who are working very hard to make sure they
• 30:48
  have, they're locked in and no one else can play.
• 30:52
  I think they're coming up with, you know, smaller language models, things that
• 30:57
  anyone can access ways, you know, like for example, national AI research resource
• 31:01
  having,
• 31:02
  access to compute, to build small models that anyone can play with, that would be
• 31:07
  good.
• 31:07
  And we should focus on that.
• 31:10
  There's also this institutional trust question, right?
• 31:14
  In that, right now, people are skeptical of big companies, but they're skeptical of
• 31:20
  traditional intermediaries and governments and other kinds of sources of trust.
• 31:25
  So how can people get the kind of confidence they need in these systems in
• 31:32
  this kind of environment?
• 31:34
  Yeah, that's difficult.
• 31:35
  And I think I don't want to drown in a sea of negativity either, because that also
• 31:40
  doesn't get you anywhere.
• 31:41
  And I think a constant suspicion of everything is not a way to live either.
• 31:45
  We do have to have some level of, if not trust, but at least a suspension of
• 31:50
  disbelief for a while while we play with some tools.
• 31:56
  I think we need more experimenting.
• 31:57
  I think we need less hype.
• 31:59
  We need to be able to, I think if we could filter out.
• 32:01
  what 99 % of the hype around these tools.
• 32:04
  We've been a place where we could actually credibly evaluate them and think about
• 32:07
  what they could do for us and what we want.
• 32:10
  The hype just makes it so difficult that we either jump into the hype and lean into
• 32:15
  it very heavily, which is, I think, a mistake, or we try to run away from
• 32:19
  anything to do with these tools.
• 32:20
  It is also a mistake.
• 32:22
  And I think so the first part of rebuilding trust is to recognize that the
• 32:28
  hype is just that.
• 32:30
  is to say if we ignored the hype, what can we do with an LLM?
• 32:35
  What can it do individually for us?
• 32:37
  And then learn from that.
• 32:39
  I mean, very wishy washy here.
• 32:40
  I don't have a good answer for you, to be honest.
• 32:43
  But the first step is to separate the hype from the tools.
• 32:47
  And that's really important.
• 32:49
  No, absolutely.
• 32:50
  And we've got to take the first steps in order to get to the next steps.
• 32:54
  That's a really important point.
• 32:55
  also struggle with the issue of trust because I find myself reflexively
• 32:58
  distrusting some of the claims made about LLMs when perhaps even I shouldn't be
• 33:04
  doing that.
• 33:04
  I should at least take them at face value to a degree.
• 33:07
  But we don't really have good methodologies.
• 33:09
  We don't have rigorous frameworks.
• 33:11
  We don't have what I would call strong scientific evidence for so many of the
• 33:15
  claims made by LLMs that there's a constant eye roll like, my God, here's
• 33:19
  another paper making claim that tomorrow will be disproved.
• 33:22
  And you know, it's just...
• 33:23
  You just have to kind of ride those waves a little bit.
• 33:27
  No, absolutely.
• 33:28
  Incredibly challenging, but also incredibly important.
• 33:31
  Suresh, thank you so much.
• 33:32
  It's been a really wonderful conversation.
• 33:34
  very much.