Latest / The Road to Accountable AI / Beth Noveck: Effective and Accountable AI in the Public Sector

Transcript

Beth Noveck: Effective and Accountable AI in the Public Sector · The Road to Accountable AI

0:01
Beth, welcome to The Road to Accountable AI.
0:04
Kevin, thank you for having me.
0:07
You were fairly recently named the first chief AI strategist of the state of New
0:12
Jersey.
0:14
Of any state?
0:15
Okay, so tell us what that means.
0:17
Well, I have to say lots of states now and in the federal government are naming a
0:21
chief AI officer or our following suit to have somebody who is responsible for AI
0:28
governance, AI use, AI education in the state.
0:31
And that very much is my role.
0:33
So I've served for the last six years.
0:35
I've had the pleasure to serve as the first chief innovation officer for the
0:39
state of New Jersey.
0:41
which was centrally about how do we use new technology to advance the priority of
0:46
serving residents of the state?
0:48
How do we use it to deliver better technology, deliver better services for
0:52
the people of New Jersey?
0:54
And since the technology du jour, of course, is AI, and it is so
0:59
transformational, we made the decision that we should have both a chief
1:04
innovation officer on the one hand, and somebody who is thinking just about what
1:09
are these new technologies.
1:11
What do they mean and how do they enhance our ability to serve residents, its
1:16
businesses, its economy, uh, create jobs as well as help the state to do what it
1:22
does for residents better and faster.
1:25
What are some of the things that you anticipate AI will be or maybe is already
1:29
being used for public services?
1:32
So already being used for.
1:34
So I think it's important to recognize, first of all, that AI has been around for
1:38
quite some time and has already been in use in public and in the private sector.
1:43
Of course, more recently, this new generation of generative AI tools that are
1:47
so good at both creating content but also analyzing content.
1:52
And whether that is numbers and data as we traditionally think about it, or whether
1:57
that's words or images.
1:59
which are also just another form of data.
2:03
What these tools I think are so transformational at doing is helping us to
2:08
get smarter faster in our institutions and do our work more efficiently, but also in
2:14
ways that are more responsive to residents.
2:17
And let me give you a few examples and explain what I mean by that.
2:20
So first of all, very important for responsible use of AI is that the AI is
2:26
really secondary.
2:28
It is the tool.
2:29
It is on tap, but not on top, to paraphrase Churchill here.
2:34
That is to say, we're not looking to eliminate anybody's job.
2:38
To the contrary, what we're looking to do is to put tools into the hands of public
2:43
servants, of public professionals, to help them do what they do as efficiently and
2:49
effectively as possible.
2:50
So for example, when we are within the New Jersey Department of Labor sending out
2:57
letters to claimants for unemployment insurance, we're now taking those letters,
3:02
70 of those templates, for example, that we use to correspond with citizens about
3:07
the service the state delivers, a very important and needed service, using that
3:11
to rewrite those letters in plain English.
3:14
So instead of sending out government speak, not that we did that before, but we
3:18
can do better.
3:19
in terms of writing in really plain English, in terms of writing in ways it's
3:23
easier for people to read.
3:25
And in fact, with really just the click of a button using these tools to rewrite how
3:29
we're talking to people, we've decreased the time by 50% that it's taking people to
3:35
read those letters.
3:36
If in the state of New Jersey, you call in to ask a question about your anchor tax
3:41
rebate that homeowners are entitled to, what we're doing is using the data from
3:45
those calls, which we can now...
3:48
better listen to and summarize and analyze using generative AI and using it to
3:53
rapidly write phone tree responses so that we're putting front and center the
3:57
information that residents actually want and need so they don't have to wait on
4:01
hold to talk to an operator.
4:04
And for those then who have a question that's not getting answered, they can,
4:08
when they do talk to somebody, we're able to answer those questions more quickly.
4:12
So that means 50% improvement now practically overnight in the
4:17
responsiveness of those calls for residents, precisely because we're able to
4:21
give people the information they want faster and easier, because it's easier for
4:26
us to write that information.
4:27
But that's just the beginning.
4:29
But let me just stop there.
4:30
Some of the very first initial uses of things like generative AI, early days.
4:36
Absolutely.
4:38
What are some of the similarities and differences of those kinds of use cases or
4:43
other kinds of use cases in a government setting compared to say a company that's
4:48
thinking about using generative AI in terms of how it communicates with its
4:51
customers?
4:53
And many of the uses are the same across organizations.
4:57
If it's writing a first draft of the CEO's speech or writing a first draft of a
5:02
minister's speech.
5:03
If it is writing a first draft of a letter, we write letters and memos in all
5:09
kinds of organizations.
5:11
If the use is, for example, to analyze data or to make an image to illustrate a
5:16
blog post, none of that's different.
5:18
This is really a next generation set
5:23
I would say both word processing tools and the calculator mushed together, um, that
5:27
are going to be helpful to anybody in every organization.
5:30
What's different is the intent in the public sector.
5:33
We're starting with the focus on how do we best serve residents?
5:37
How do we best serve the public and how do we advance the public interest?
5:42
So we are, so that intent, and I would like to think that both the private sector
5:47
and the public sector companies and governments can and should
5:52
have the public's interest at heart.
5:54
So if you come to the equation saying, how can I cut the most costs, I'm going to end
6:02
up potentially in a different place than what I'm saying is, how can I deliver the
6:06
best customer service?
6:07
So I would like to think that the best companies are asking that latter question,
6:12
not just the former question.
6:13
If I'm coming to the equation with the question, how can I use these tools to
6:18
make my workers more productive?
6:20
so that they can do more and do better.
6:23
That's a far, I think, better attitude than, how can I use AI to fire my workers
6:30
and get rid of them?
6:31
So it's the intent that you bring to it.
6:33
In government, the public interest, and I think that goal of augmenting workers
6:40
rather than replacing them is really core to the values that we bring to the
6:44
equation, but that doesn't have to be limited to the public sector.
6:49
Does it change how you think about responsible or ethical AI?
6:54
It does in the sense that I really think that government is a really, really
7:01
important user of these tools.
7:03
There's obviously a lot of concern given what we've seen from social media.
7:09
And I would say the kind of degradation of the quality of social media, the
7:14
proliferation of disinformation and misinformation that's given pause to a lot
7:20
of us about private ownership.
7:23
of the platforms that are effectively our public square.
7:27
It's where we're spending our time, it's where we're socializing with one another,
7:31
it's where we're building community, and yet all those places are privately owned,
7:35
and we've seen the consequences of that.
7:38
If the CEO of the company, if the board of the company makes the decision that we're
7:42
going to, for example, fire all the content moderators, or we're gonna let
7:46
hate speech proliferate on the platform, it really degrades the quality.
7:50
So I think that's given people a lot of pause as we now embrace this new set of
7:56
tools.
7:57
What does it mean that these powerful technologies we're going to use for
8:00
creating content, for editing content, for education, for healthcare, for all these
8:06
important uses are going to be in the hands of a couple of companies and a
8:11
handful of private individuals.
8:13
So I'm pretty excited about government as a creator and user of AI because of the
8:19
role that it can play in really trying to both design and build and use products for
8:26
public purpose.
8:30
What do you see in general as some of the most important challenges for AI
8:35
governance or responsible AI, or I like to talk about accountable AI?
8:38
Choose your phrase.
8:41
But what do you think is most important?
8:42
what do you think are the differences, or I'm curious about if I'm allowed to ask a
8:46
question back, how do you think about the difference between responsible and
8:49
accountable?
8:51
Absolutely.
8:52
I use accountable to emphasize the connection between the activity and the
8:57
implications and the response.
9:00
That it's about ensuring that there is an information flow and that leads into an
9:06
understanding of the issues and that leads into a set of obligations, which might be
9:11
internal or they might be external.
9:14
And people can put that under these other rubrics as well.
9:20
hear ethics or even hear responsibility, they think it just means, oh well, just do
9:26
the right thing, as opposed to having a sense of putting all the processes in
9:30
place that it takes to do that effectively.
9:33
Yeah, I think we're all trying to figure out right now what is the right thing with
9:37
technologies that are changing so very rapidly and what that looks like.
9:43
But it obviously starts by having some guidance as people begin to use these
9:50
tools.
9:51
So the idea of having policies, even if they're interim policies that will change,
9:56
having some guidelines for workers.
10:00
so that they can go out and try these tools.
10:02
And that's the second piece of it.
10:04
Responsible use, the emphasis also has to include use.
10:08
And we're not going to understand what it means to be responsible unless we've
10:12
actually tried these tools and understand what they're good at and where their
10:17
limitations are.
10:18
It's very hard to know how to respond to and mitigate risk for something that I
10:23
don't understand.
10:24
And frankly, I have been in the room with regulators, people responsible for...
10:29
deciding what the fate of these technologies will be.
10:32
And when I've asked, how often do you use Cloud, ChatGPT, Google Gemini, et cetera,
10:39
and no hands go up, it gives me great pause to think about making the people
10:46
making the rules of the road not having the experience.
10:49
It would be like trying to create the traffic laws if you've never driven a car.
10:54
So we need to have some experience.
10:56
That means using these tools.
10:58
And to be able to use them, we do have to have some guidelines about what is and
11:02
what isn't safe as driven by those who have some experience.
11:08
But the other piece of this is education.
11:10
So use is part of that.
11:12
And education and training, because it's not enough just to have a policy, that is
11:18
to say a piece of paper or a document, I need somebody to really show me.
11:21
So in my case, I have a teenager at home who does that work of showing me what it
11:25
is.
11:26
I should do with these tools and all the cool things that I can be accomplishing.
11:31
And I think it's so important in organizations, whether private or public,
11:36
to ensure that we have a robust program for training workers, up-skilling people
11:41
to use these new technologies so we can understand what does it mean for each of
11:46
our jobs day to day.
11:48
And that's going to look different depending on what our work is and what we
11:51
do, but educating people about what those uses are.
11:56
Again, what the benefits are and the potential uses, what the risks are and how
12:00
do we mitigate them has to be both policy and practice.
12:04
That's why in New Jersey, Governor Murphy has been very forthright about saying,
12:08
we're going to train all of our public sector professionals in the use of AI so
12:14
that people can begin to understand how do we use these technologies for good.
12:18
And finally, I will say that responsible use, as with any technology,
12:22
has both a set of negative prescriptions, all the things we don't wanna do, that is
12:27
to say, don't put personally identifiable information into these tools, don't type
12:31
in anybody's social security number, and don't type in your own.
12:36
But it also has a set of positive prescriptions.
12:38
And I think that's important is to recognize that accountable, responsible,
12:44
beneficial use is about asking and answering the question, how can we use
12:49
these tools for good?
12:52
not just what do we not want to do, but what can we do that actually is going to
12:57
lead to improvements in people's lives.
13:01
That's interesting.
13:01
It ties into your innovation role.
13:05
How do you create an environment in government that overcomes the natural risk
13:09
aversion?
13:09
Because if something gets deployed and there's a problem, there's going to be
13:13
criticisms.
13:14
So how do you create an environment where people are willing to try those new
13:19
technologies, use the AI for the benefits, address as much as possible the concerns,
13:25
but not be too afraid?
13:27
So I think different institutions have gone about this differently.
13:30
You know, in the U.S.
13:31
Congress, what they did was have a pilot group, a sort of cohort of a few hundred
13:39
people who were given licenses to chat GPT on the condition that they would report
13:45
back about how they were using them and sit down and have a conversation and
13:49
discuss what's working, what's not working again, what are the beneficial uses,
13:53
what's not working again.
13:55
what's not working well, et cetera, and educate one another.
13:58
So they started with a pilot, essentially, and said, we're not gonna give it to
14:01
everybody, but we're gonna give it to a few people.
14:03
And then actually what they did was take back those licenses and now give it out to
14:07
a few hundred engineers who work for Congress to let them play with the tools,
14:12
see what they can actually do with it.
14:13
Other organizations with AI and other technologies have created what they call a
14:18
sandbox.
14:19
They create a secure environment behind a firewall.
14:23
They have the appropriate licenses already procured.
14:27
And they create literally a physically safe space in which you can try some of
14:32
these tools, and again, with policies and guidance, et cetera.
14:36
And again, what we've done in New Jersey is to put out this policy.
14:41
But we went beyond just having a piece of paper that says,
14:45
be careful in how you use these tools.
14:47
We laid out very specifically in the policy, and that's quite unique.
14:51
It's a longer document because it says, here are examples of how you could be
14:56
using these tools to serve residents.
14:58
Here are things you should not be doing in terms of privacy, in terms of
15:03
cybersecurity, in terms of personally identifiable information, and in terms of,
15:08
again, safeguarding the public interest in how you do your work.
15:12
We provided training video to go with it that illustrates the policy and actually
15:16
shows you what we mean.
15:19
Not enough, again, to explain people, use these tools responsibly if you've never
15:24
even tried them before.
15:26
And again, now we're rolling out a government-wide training for everybody to
15:31
help ensure that we have those protections in place.
15:34
That goes again with other sorts of things like signing terms of use, procuring, you
15:41
know,
15:41
the appropriate technologies so that to help people in making the decision about
15:46
which tools should I be using that are actually safe.
15:49
And lots of kind of guidance and advice about, again, how to use these tools and
15:56
what to do with them as we encourage people to try and see what works.
16:02
What do you think is the biggest gap or the biggest concern with regard to
16:08
problems that might occur with these kinds of deployments?
16:11
Well, you're absolutely right that there will be the things that we, there will
16:16
inevitably be bumps in the road as there have been with every kind of technology.
16:21
And I want to double down on emphasizing that while AI and generative AI are a
16:29
powerful set of data processing tools, there are things that they can do that no
16:33
other technology before them has been able to do and they are unique.
16:40
in their power and capability.
16:42
They are also just another new technology.
16:45
And we have encountered the same challenges before as we've embraced any
16:50
kind of software or technology where we have had to ensure that we mitigate the
16:57
risks from hacking and cybersecurity from again, that we have.
17:05
processes in place to review the technologies and make sure, and now I'm
17:09
going to get to the answer to your question, that the first and foremost we
17:13
are defining the problem we're trying to solve and then picking the designing the
17:19
process and the technology that will lead to the solution to that problem.
17:25
We've seen the floor littered with apps that do not solve
17:31
anybody's problem, frankly.
17:33
They just take up space in your phone and they don't do much good at all.
17:37
But in other words, technology that's often not really well-suited for purpose
17:43
is badly designed, makes people's lives worse in terms of having websites that are
17:48
more complicated and harder to use.
17:51
So floor is littered with bad design and bad tech.
17:55
And as with these tools, as with any other,
17:57
got to start with what's the problem we're trying to solve and then let's figure out,
18:00
you know, is this the right tool for the purpose and how do we use it?
18:05
But in the first place, you know, again, if we start with what I would call low
18:10
risk uses, that is to say, and I don't mean this in the European legal sense or
18:17
any formal sort of term of art, but what I mean is if we're starting with largely
18:22
internal uses.
18:24
rather than public facing ones, that's a pretty good way to start to figure out
18:29
what is it that I can do with these things and avoid the missteps that we've seen so
18:34
many of, the lawyer who files the brief, forgetting to remove the fact that says
18:41
this was written by Chad GPT, or the person again who writes the paper that so
18:48
obviously has the hallmarks of having been written.
18:51
by generative AI and never read by anybody.
18:54
So those missteps we will get better at if we practice on uses that are internal, as
19:01
in writing the first draft of a memo that is gonna be internal to my office rather
19:08
than using it on things that might be public facing in the first instance.
19:12
And then.
19:14
of sense.
19:16
You also have an academic role as a professor at Northeastern University.
19:20
I'm curious how your
19:22
matching dueling bookcases here.
19:25
Exactly.
19:26
How does your academic background as a law professor feed into the work that you do
19:31
in New Jersey?
19:32
Oh, well, I think in a variety of ways.
19:37
So I'm very, very lucky to be able to wear these two hats.
19:42
I mean, on the one hand, what it allows me to do is to have an excuse to keep up on
19:47
the latest in research.
19:49
And this is a field that's changing so fast.
19:51
So being able to be aware of what are developments with these technologies, what
19:57
are their capabilities, what are they doing to be able to look around the world
20:00
and say, not just what are we.
20:01
doing in New Jersey, but what's happening in other states, other countries, how are
20:07
other kinds of institutions, most executive, but legislative and judicial,
20:13
using these tools and to bring some of that knowledge back and to bear in the
20:17
work that we're doing.
20:19
In my new role in particular, more so than before, I'm very excited that I will have
20:25
more opportunity to be able to collaborate and work with universities.
20:30
And so my background as a teacher and as a researcher is helpful here, God bless you,
20:35
for exploring sort of ways in which we can get students involved in building
20:40
technology and doing projects with governments, which is something I spend a
20:45
lot of my time thinking about and doing.
20:47
Because I think one of the things that generative AI does in particular by
20:51
bringing down the cost and the time it takes to write software code.
20:57
is making it easier and will make it easier for students and others who may not
21:03
have years of experience to be involved in things like the development and delivery
21:09
of digital services for companies and for nonprofits and for governments.
21:13
That's gonna be a great experience for students, but it's also gonna be very
21:19
helpful for institutions.
21:21
So on my...
21:22
The professorial side of things, that's what I do, is I work with students on
21:28
building generative AI projects for public institutions, which is a great experience
21:33
all around.
21:34
Students learn how to take something from idea to implementation in collaboration
21:38
with the communities they're trying to serve.
21:42
Governments, nonprofits, and others who otherwise wouldn't have the ability to
21:46
benefit from these technologies and don't necessarily have the personnel.
21:51
who currently work for them with this training get to benefit from students with
21:55
the latest training in AI and computer science.
21:59
Who do you look to or where do you look to, whether it's states, other governments
22:04
or companies for models for effective implementation of AI?
22:09
Hmm, it's a great question.
22:12
Um, you know, I'm completely biased because I do love the work that our teams
22:20
and that my colleagues in New Jersey are doing after many years of having an
22:26
innovation team and really doubling down on collaborating with our agency
22:32
colleagues.
22:33
to build digital services and use new technology to serve residents using not
22:39
just technology, but a human centered design.
22:43
That is to say, listening to and engaging with residents to be able to hear from
22:47
them, what is it we should be building and how can we make it intuitive and easy for
22:51
them, using data to help drive that decision making and design.
22:57
I've been very inspired by some of the things that are getting built here, and
23:01
I'm really always excited to evangelize about a lot of those examples.
23:05
That said, I just came from a few minutes ago, having had the pleasure of the tables
23:14
being flipped, and that I was the interviewer, and the interviewee was
23:19
Minister Audrey Tang of Taiwan, who is the digital minister.
23:25
And what she has been doing in Taiwan has just for years now to embrace the use of
23:31
new technology.
23:33
First and foremost, and most excitingly, to involve citizens in decision making
23:39
processes.
23:41
It's really exciting.
23:42
So her story by itself, she was not a government professional.
23:46
She was an activist who actually participated in occupying the parliament.
23:52
and later became an advisor to a minister and then became the minister herself but
23:56
came in from the outside with the knowledge of new technology began to
24:00
educate the government not just in how to use technology but more importantly in how
24:05
to collaborate with people outside of government how to work with the civic
24:09
community how to work with the business community how to work with others who are
24:14
potentially more innovative and bring that technology into government so as she
24:18
explained a few minutes ago
24:21
Taiwan's very successful contact tracing program during COVID was entirely built by
24:27
people from outside of government that they then just adopted as the standard for
24:32
what they did.
24:34
The platform that was known, still is known as VTaiwan, where they used a
24:40
variety of different online platforms to get citizens engaged in collaboratively
24:45
drafting what turned into 26 pieces of legislation.
24:50
was all developed by activists from outside, and then the government embraced
24:54
the results of that.
24:56
So I think Taiwan is very sort of forward leaning, and now they've been doing work
25:01
to use AI enabled platforms to go out and ask citizens about their views on AI.
25:09
And then the AI is very helpful, of course, in synthesizing and summarizing
25:13
what they're hearing from people.
25:15
I would say the other place is really Iceland.
25:19
Iceland has been for a long time a leader in digital government, but also in that
25:25
kind of community-engaged approach with its project Better Reykjavik, which over
25:33
20 percent of the population actively participates in proposing policy ideas to
25:40
the city of Reykjavik, to the national government.
25:44
which then gets acted upon.
25:46
So they have just this, in both places, a much better oiled machinery for engaging
25:53
citizens in government beyond election day.
25:57
So those are the places that I've always looked to as models that I'm really
26:00
excited about that use a lot of free and open source software to do this.
26:05
But every state in the United States and so many cities, every time you turn
26:10
around, you'll find pockets of
26:13
innovation that's happening in all over the place.
26:19
There have, though, been scandals in various countries with the use of at least
26:24
algorithmic decision-making systems, like the UK post office scandal, the
26:28
Netherlands, and other places where governments have used data and algorithmic
26:34
decision-making in ways that have been harmful or discriminatory.
26:38
So what does it take to ensure that our governments have that more openness that
26:45
you're talking about in their deployment of these technologies?
26:48
Yeah, well, that is, you know, those, there are some obviously truly frightening
26:54
examples.
26:56
The UK post office one, for those who don't know, I think it's now common
27:00
knowledge here, but that was the case where the UK post office employed a
27:08
technology system.
27:10
I think Toshiba was the vendor, if I remember correctly, and it was falsely
27:14
accusing,
27:18
what they call, what we would call a deputy postmaster, I guess, in this
27:22
country, sort of junior postal officials of committing fraud.
27:27
And a lot of people got caught up in this set of accusations.
27:32
And then of course, some of those people went to jail.
27:35
One of them even committed suicide.
27:38
And there was clear evidence that the technology system was just spitting out
27:43
bad results and it was completely wrong.
27:46
And yet, and here's where the scandal comes in and why I wanted to explain this,
27:51
in the end, it wasn't about the bad tech.
27:54
It was about the fact that the leadership doubled down on, dug their heels in and
27:59
refused to listen and doggedly pursued these persecuted people and pursued these
28:05
prosecutions, even in the face of facts that told them this was wrong.
28:13
So everywhere we're going to have problems of people in power who do not want to
28:17
relinquish their power.
28:19
And yes, technology will be a help meet to those sets of practices.
28:24
We know that there are both governments and companies who will want to use
28:28
algorithmic decision making, whether it's to bring down the cost of hiring workers,
28:32
which is, you know, a very important task that companies especially need to engage
28:37
in.
28:38
or making the decision about who gets paroled or who gets prosecuted, there's
28:42
going to be so much pressure to use these tools to realize efficiencies.
28:47
When we don't know what's going on under the hood, to use your word, there's no
28:52
accountability for the technology.
28:55
And the intent with which we approach these is going to be really important.
28:59
This is why I think it's very pathbreaking that in Europe they've said...
29:03
look, when it comes to these kinds of systems that are making decisions that
29:07
affect people's freedoms, that affect people's education, that affect people's
29:11
life outcomes, we are going to scrutinize these things and regulate them really,
29:18
really carefully.
29:19
It's not the writing the first draft of your memo with ChatGPT that we're worried
29:24
about.
29:24
It's the sending people to prison with the machine learning algorithm that, or
29:30
depriving people of insurance.
29:33
or depriving someone of a mortgage or a college degree because an AI made a bad
29:38
decision because of bias training data or just a bad piece of software that doesn't
29:48
do what it's supposed to do.
29:49
So coming back to a lot of your risk scenarios you were asking about earlier,
29:53
it's in these are the contexts that we're really worried about and that we are gonna
29:57
need to regulate.
30:01
Absolutely.
30:02
So on the more positive side, how do you measure success?
30:07
Or how should a government agency measure success when it's looking at something
30:11
like an AI deployment?
30:14
Well, we always need to measure success based on the lives of the people we
30:17
improve.
30:18
Are we making somebody's life better as a result of this?
30:21
That's got to be the question.
30:23
Um, and that should be the North star in terms of how we think about the deployment
30:27
of these technologies.
30:28
Uh, so we want to know, for example, when the New Jersey Department of Labor deploys
30:34
mycareer.nj.gov, it's new career coaching and training explorer tool set.
30:43
Is it helping workers to make a more informed decision about what training
30:48
course they should take?
30:50
That project uses machine learning very uniquely among government websites, not to
30:56
provide one-size-fits-all information as most governments do, but to actually give
31:01
you personalized recommendations about your job search, about careers that could
31:07
be a good fit for your skill set.
31:09
and about the training that you might want to take to prepare you for that next
31:13
career.
31:14
So it's using machine learning to match your skill set from your resume to
31:21
in-demand job data to be able to provide those recommendations.
31:27
Is that helping someone to get a job?
31:29
Is it helping someone to take a course that will prepare them so that they don't
31:33
lose their job to an AI algorithm at some point?
31:38
Um, is it making someone's life better when we do that?
31:41
When we use chat GPT to redraft a letter that we send to a claimant for
31:46
unemployment insurance, is it helping you get that benefit faster so that you can
31:51
pay your rent so that you can eat so that you can feed your family?
31:55
That really has to be, um, the metrics we have to use.
32:00
Obviously in the shorter term, we can ask questions like.
32:04
you know, did we decrease the time it took to open the letter, or did we answer the
32:08
phone better, faster, or did it take less time to read this website?
32:12
Were people happier?
32:14
Lots of things we can and we need to be measuring, and the good part is AI,
32:18
because it's so fast at processing data, can actually, I hope, help us to do that
32:25
kind of measurement, which has eluded us in government for a long time, to do that
32:29
better and faster.
32:32
Last question, what can citizens or companies that are developing technology
32:38
in this area do to help and promote governments taking these kinds of steps
32:44
that you're talking about?
32:46
So I think, you know, first of all, making those demands, the more pressure that's
32:52
put on governments to say, you know, New Jersey's providing personalized career
32:58
advice.
32:58
Why isn't my government providing personalized career advice?
33:03
You know, I can, I don't have to wait on the phone, you know, to get this benefit.
33:08
You know, why isn't my government doing that?
33:10
So I think creating that demand is really, really important.
33:13
Number one.
33:15
Number two, coming back to this idea of the importance of collaboration and
33:20
engagement, I do think that there's going to be a lot of receptivity now as
33:25
governments are hungry for talented people who understand something about these tools
33:32
for collaboration.
33:34
So there's been an effort at the federal level, for example, and in various states
33:39
to bring people into government.
33:42
who have different kinds of skill sets and doing a stint in public service is, I can
33:48
say, first-hand experience is, you know, it's been the most rewarding experience of
33:54
my life and I really recommend it to anybody.
33:57
And there are many ways to serve, not just at the federal level or the state level,
34:02
but for your town, for your community.
34:05
You know, there isn't a town out there now who wouldn't like some help, for example,
34:09
making a chat bot, as they've done in
34:12
Prospect Park, New Jersey, which is a tiny little town in New Jersey but which has a
34:18
very forward leaning mayor of at the time when he was appointed the youngest mayor
34:22
ever appointed won't surprise you given his tech bent Who has worked with friends
34:27
and collaborators to create a chatbot?
34:29
So even in this tiny little town a resident can get answers to their
34:33
questions about the city services 24-7 without having to go to City Hall or wait
34:39
online
34:40
or go nine to five during the work day, they can get better served.
34:44
So I think there's lots of opportunities.
34:46
They were approached by somebody who said, hey, we're here to help.
34:49
What can we do?
34:51
And I think there's lots of opportunities like that.
34:53
And I would just encourage people to get involved and bring their know-how to bear.
34:58
Thanks so much.
34:58
It's always wonderful talking with you.
35:00
Kevin, thank you for having me and being so interested in what the public sector is
35:04
doing.
35:04
It is the largest employer in the United States and the largest purchaser of
35:09
technology in the country.
35:11
So I think what it does is going to be important for all of us as citizens, but
35:19
also across sectors.
35:21
A government being a smarter consumer and user of technology, I think will.
35:26
have a follow on effects for every business, for every sector, and every
35:30
industry.