Latest / The Road to Accountable AI / Kevin Bankston: The Value of Open AI Models

Transcript

Kevin Bankston: The Value of Open AI Models · The Road to Accountable AI

0:09
Welcome to the Road to Accountable AI.
0:12
I'm Kevin Warbeck, Professor of Legal Studies and Business Ethics at the Wharton
0:15
School of the University of Pennsylvania.
0:17
For decades, I've studied emerging technologies from broadband to blockchain.
0:22
Today, artificial intelligence promises to transform our world, but AI won't reach
0:27
its potential without accountability, mechanisms to ensure it's deployed in
0:31
responsible, safe, and trustworthy ways.
0:34
On this podcast, I speak with the experts leading the charge for accountable AI.
0:40
Whether to restrict open weight foundation models is a big topic of discussion
0:44
in AI safety and policy circles.
0:47
Thanks But not one that the general public and business community are
0:51
necessarily focused on my guest, Kevin Bankston from the center for
0:54
democracy and technology talks about why this issue is so important,
0:59
both to the future of innovation in AI and to our ability to address.
1:04
Serious AI risks, as you'll hear, Kevin has strong views and in full disclosure,
1:09
I signed on to an open letter that CDT wrote on these issues, but in our
1:13
conversation, we cover the arguments on both sides of the issue and why
1:17
this topic is actually so important to understanding what the future
1:21
path holds for the development of AI.
1:23
Kevin, pleasure to have you on the podcast.
1:26
Before we get into talking about open weight models, just a quick Tell us
1:31
a little bit about your background and for those who aren't familiar
1:35
what CDT is, what's the work that the organization is doing on AI?
1:39
Sure.
1:40
I'll start with CDT.
1:40
CDT is, I would say, the premier internet and technology policy think
1:46
tank and advocacy organization in DC.
1:48
We've been around for, I want to say, almost 30 years at this point.
1:52
Non partisan, non profit, focused on protecting civil
1:57
rights and civil liberties.
1:59
In the digital world that encompasses a whole lot of things.
2:03
It's a broad spectrum organization, but obviously right now AI is at the forefront
2:08
of a lot of technology policy discussions.
2:10
And I, um, with my colleague, Miranda Bogan co founded
2:15
the new AI governance lab.
2:17
At CDT, um, which is focused on helping develop best practices and standards
2:23
around evaluating and mitigating AI risks to help build a technically informed basis
2:29
for eventual legislation and regulation, taking advantage of not only my 20 odd
2:37
years in civil society at organizations.
2:42
Like and including CDT.
2:44
After that 20 odd years, I was approached by Facebook, now Meta, to basically
2:51
help them figure out their path forward on responsible AI development,
2:55
which was an intriguing possibility.
2:57
And so I spent about four years building their AI policy team, being
3:02
one of the founding senior leaders of their responsible AI product team,
3:07
which consults with all the other product teams on their AI products.
3:10
Thank you And did a lot of good work, but obviously working in a big company like
3:15
that in a lot of ways is very stressful and challenging and time consuming.
3:20
And I am an, at this point, a late middle aged fella who just, more
3:24
accurately, my wife just had a baby.
3:27
And I decided for a variety of reasons, but especially including
3:30
that one, I was ready to leave.
3:32
The corporate world and come back to civil society and apply what I had learned.
3:36
And also what Miranda, as my colleague on my team at Metta had learned
3:41
around responsible AI development.
3:42
So now Miranda who did trail trailblazing groundbreaking work at
3:49
Metta in terms of addressing issues of race and gender bias in ads delivery.
3:54
She is the director of the new governance project at CDT.
3:57
While I'm a senior advisor to that project and to the leadership at
4:00
CDT on other AI policy issues.
4:03
I currently teach the First Amendment and copyright in regard to AI technology
4:09
as an adjunct professor at Georgetown.
4:11
Those are issues that CDT has not spoken greatly about.
4:15
We do address the First Amendment in regard to this issue a little bit in the
4:19
comments we're going to be talking about.
4:21
But speaking generally, if I talk about the First Amendment here, attribute
4:24
that to me as Kevin the scholar, not necessarily CDT's positions.
4:29
Great.
4:30
So this is why you are a perfect person to talk about these issues.
4:33
I want to ask you in particular about the debate that's happening now
4:37
around open weight foundation models.
4:39
So first, for those who aren't necessarily familiar, what are we talking about here?
4:43
So this will require a little bit of a history lesson about What
4:47
open source software is before we get to what open source AI is.
4:52
So in the nineties, you began to see the evolution of this new form of.
4:59
Software development and licensing around this open source concept where.
5:05
You would have companies or even voluntary projects with a bunch of contributors
5:12
developing their code openly, as in collaboratively, or when publishing their
5:17
programs, also publishing the source code itself so that other people could use that
5:23
to build their own software, to modify it for their own uses, et cetera, et cetera.
5:29
Usually this is accomplished through something called an open source license,
5:33
which basically says, You can use this and there are no use restrictions on it.
5:38
We are essentially.
5:39
Although we still own the copyright in this technically, we are
5:41
licensing it broadly for public use.
5:45
So, why is this important and, speaking generally, good?
5:49
Certainly the internet and digital technologies built around and
5:53
on the internet rely an enormous amount on open source software.
5:58
At this point, 96 percent of all codebases include some open source code.
6:03
It's the original open sourcing of the Netscape browser in the 90s, which
6:07
led to the open source Mozilla Firefox browser, which was a predecessor to
6:12
later the Google Chrome open source browser that enabled competition
6:16
so that we didn't all just have to use Microsoft Explorer on the web.
6:21
Now, I want to be clear, open source is not and will not be Always a
6:26
silver bullet for competition.
6:28
And it can be leveraged by already powerful stakeholders
6:31
to make them more powerful.
6:32
Certainly it's enhanced Google structural power, the fact that it controls
6:36
ultimately the development of Android, for example, but then again, that, that
6:41
power has also helped counterbalance the power of other people, other companies,
6:46
like your Microsoft's and your Apple's, and enabled a large number of hardware
6:50
vendors to compete that wouldn't.
6:51
Have had an operating system to use and it's complicated ecosystem.
6:55
But ultimately open source in the context of regular software has
6:59
been a boon for competitiveness.
7:00
Also a boon for security in the world of open source.
7:04
They say enough eyeballs make all bugs shallow.
7:08
This is because if you have.
7:10
Your source code interrogable by anyone.
7:13
That means you have a lot of eyes looking at it and finding flaws
7:15
and helping you patch those flaws.
7:17
That's why, for example, we have the security world has generally
7:21
been supportive of the open sourcing and wide availability of.
7:27
Penetration tools for breaking into computer systems, not because
7:30
they want to help people break into systems, but because they want to
7:33
help people detect how to break into systems so they can fix those things.
7:37
And so speaking generally, open source has also been a boon.
7:40
For our overall technical security.
7:44
Now we have to get to AI though, and open source is a little bit of a misnomer
7:49
when talking about AI for reasons I'll get to first, I'll have to back
7:52
up and say, what are we talking about when we talk about AI very generally.
7:56
When we're talking about AI, we're talking about machine learning,
7:59
which is using software to look at vast amounts of data to find
8:02
patterns in it that are useful to us.
8:04
In the current context, we are usually talking about large language
8:09
models or other generative models.
8:12
And to break that down as simply as possible and get to the answering
8:14
the question of what are weights.
8:16
I'll just say to make a large language model, you take a whole lot of data.
8:22
Often everything you can scrape from the web, data you've
8:26
licensed from other vendors, a large amount of language, right?
8:30
A large amount of language and feed it through an algorithm that creates what
8:36
is essentially a massive mathematical space that sort of summarizes all that
8:42
data, not directly, but rather each.
8:45
Not even each word, each part of a word, and they're called tokens in this
8:49
context, gets its own little spot in this sort of semantic map of all of that
8:55
content, so that, for example, as it has its ingested more and more data, it
9:02
begins to not understand, but be able to place close to each other related
9:07
semantic concepts like dog and puppy, for example, are close together in this map.
9:13
What happens then is once you've trained that model to create that mathematical
9:19
map, which is what we call weights, Because the, to put it in non technical
9:25
language, like all of that semantic content has been weighted to some extent
9:29
or another in this mathematical model.
9:31
You need the weights to be able to infer anything for the model
9:35
to actually give you an output.
9:38
Open weights, and so now the analogy to open source comes in.
9:43
There, as with regular software, there's both open and closed approaches.
9:48
The closed approach is, for most of its models, The chat, a good example
9:57
is OpenAI and the way it treats most of its most powerful models, which
10:00
is it is available through them.
10:05
You have to either go to their website and pay the 20 bucks to access their
10:11
user interface for chatting with ChatGDP, or you're a developer who
10:17
pays to access their API, their application programming interface.
10:21
To build apps or services, internal or external on top of their model.
10:26
Either way, though, they have centralized control over the model.
10:32
Users and developers do not have access, direct access to the weights.
10:37
And yeah, so that's closed AI.
10:39
Open weights AI, and I hesitate to call it open source for reasons I'll explain,
10:44
is when you publish, at the very least, the model software and the weights.
10:52
So that another developer could deploy that on their own infrastructure
10:59
with their own modifications.
11:01
So in many ways it is similar to open source software because
11:06
these open weights models are typically offered under a license.
11:09
That either that, that allow for free redistribution and or don't have use
11:15
restrictions and or otherwise give you freedom to play with the model
11:20
and build derivative models in a way that you can't do with a closed model.
11:25
I don't call that.
11:26
Open source for a few reasons.
11:29
Um, one, most of the components of the model are not in the form of source code.
11:36
And in fact, this critical heart of the model, the weights, is not really
11:40
interpretable by a human at all.
11:42
Um, but I also don't call it open source automatically because
11:46
it depends on the license.
11:47
Speaking generally, an open source license is one that does
11:52
not have use restrictions on it.
11:55
And yet, as we'll talk later, there are emerging some new kinds of
12:00
licenses that, for AI safety reasons, do actually attempt to restrict
12:04
the allowable uses for a model.
12:09
Okay, so let's get into the policy issues.
12:11
It sounds companies can decide, open, close, as they do with software.
12:16
What's the concern?
12:17
There are a number of concerns, but I'd love to start with the benefits.
12:20
Because really, the policy question requires us.
12:24
To weigh the benefits and the risks.
12:27
And I think the benefits, although there are some ways that open source software
12:31
and open weights AI are not identical, there are a lot of ways that they are
12:36
similar, particularly in their benefits.
12:39
And I break these benefits down into three basic categories.
12:41
One is simply distributing power, whether in the market or in the
12:47
culture, which I'll explain basically.
12:50
As with open source software, we expect and we currently see open weights AI
12:57
being a strong competitive pressure against open against closed providers, not
13:03
least because it is free, we are seeing very fast diffusion of the technology.
13:08
We see very large enterprises adopting it, including for internal
13:12
uses like Dell and Wells Fargo.
13:14
There's a great quote from Adele.
13:16
Senior VP.
13:17
That's basically like, why would we pay for a general purpose model that
13:21
does not know much about our company?
13:23
And if we want to teach it that we have to upload all of our private
13:26
documents to their cloud and we have to pay for it, or We could use this
13:32
free software and create our own bespoke model for our own purposes.
13:37
That is more efficient, more tailored to our needs, et cetera, et cetera.
13:41
And so you're seeing that the second big category of benefits
13:45
is simply the catalyzing of.
13:49
Innovation, not only innovation in AI, but in all fields that can leverage the AI.
13:54
The first LLMs that were built were open research.
13:59
They were openly published the Google, the Google research scientists who
14:04
created the first LLMs in 2017 ish.
14:07
That is the root of the explosion of AI innovation today is the fact
14:10
that research happened in the open.
14:13
Thanks to open development.
14:14
You also see an explosion of developers.
14:17
Taking open models and then using them to build smaller, more
14:21
efficient, customized models.
14:23
Um, Build models that are small enough to run locally rather than
14:27
in the cloud, which has both privacy benefits and environmental impact
14:32
benefits and just bandwidth benefits.
14:34
Like you don't have to spend all that network infrastructure.
14:37
Research around AI models also enables a wide swath of security and safety
14:42
research that can't happen effectively.
14:44
with closed models and can enable the faster development of including
14:49
the faster development of tools to detect and prevent bad things.
14:52
There's another way in which it can help security rather than hinder it.
14:56
And then all the other fields that leverage this AI,
14:59
they will also move faster.
15:00
And so there is a benefit of simply speeding up the process of
15:04
innovation and opening that process to a lot more stakeholders as well.
15:09
And then finally, there's the, the transparency benefits.
15:12
Which also brings security and accountability benefits.
15:14
Like with regular open source software, with enough eyeballs, a lot
15:19
of those problems will be discovered and can be remedied because the
15:24
whole world essentially is your red team, the team that's testing it.
15:27
There's a lot of research that's happened that couldn't have happened
15:30
without open source models around guardrails, the guardrails that
15:34
companies use to try to prevent bad things from happening with their models.
15:37
A lot of those, the challenges with those guardrails have been
15:40
discovered through open source.
15:42
Models, um, develop in a way that actually reflects on.
15:48
The flaws of the closed models as well, because they're similar
15:51
architectures, the closed and the open models, things that you learn about
15:55
open models can be applied and you're thinking about closed models as well.
15:58
So it's a way of even getting more transparency into the closed models,
16:01
even though we don't actually have literal transparency into the closed
16:04
models and it's enabled research around bias and discrimination that
16:09
wouldn't have been possible without access to the model and the weights.
16:13
And one of the things we've discovered in all that testing is, and this is
16:18
Problematic and worrisome, but also relevant to the analysis of whether open
16:22
source is worth the risks, considering the benefits is the guardrails that
16:26
we are currently using to try to.
16:29
Basically say no, have a model say no to you.
16:31
When you try to do something dangerous or problematic, those are very fragile
16:36
and they're fragile, whether or not you are using open or closed models.
16:40
And we wouldn't have known that without research that was enabled by open models.
16:45
This goes to an important concept that we'll need to talk about when
16:48
we talk about the risks, which is what is the marginal risk of open
16:53
source or open weights models.
16:56
Compared to other technologies that are available, whether it's the internet
17:01
itself or closed models that are also available to be used, because
17:06
if there's not a big, or there, if there's not a meaningful marginal
17:11
risk that is differential risk between those two things, there's no reason
17:15
to target open source specifically.
17:18
For restriction and we should probably think about a different
17:21
approach Okay Sounds wonderful Why then are there concerns?
17:28
Let me ask you in a more targeted way Ntia national telecommunications
17:32
and information administration in the commerce department Launched
17:35
this request for comment in early 2024 about open weights models.
17:41
What, why did the government feel the need to even ask whether these should
17:47
be allowed to exist in a free form?
17:50
Yeah, I think there are a number of different categories of
17:53
risk and a number of different.
17:54
phases of concern that we've gone through at this point in the past two
17:58
years, since chat, GTP dropped for real, I'd say the first category I
18:05
would call emergent existential risks.
18:09
I would say these were very much at the forefront of.
18:15
Initial thinking about the dangers of AI because there is a community, we'll
18:20
call them the AI safety community, although there are plenty of other
18:23
different types of AI safety folks and stakeholders, there is a community
18:28
around AI safety that essentially.
18:31
Evolved over 20 years from work being done in the Oxford philosophy department
18:36
around potential existential risks that humanity might face and one of
18:41
those these people have theorized for a while now is the possibility of some
18:46
out of control super intelligent AI that we lose control of and that does
18:52
something catastrophic to harm humans.
18:56
for its own reasons or because it misunderstood its instructions.
19:00
And I don't want to denigrate the people who are raising this concern.
19:03
Speaking generally, I think these are very intelligent, very well
19:07
intentioned, sincere people who want to see these concerns addressed and
19:13
don't want to see AI destroy the world.
19:17
The more likely threat than the sort of vague super AI is the possibility
19:23
that these systems, as they get more intelligent, could significantly
19:28
aid adversaries, whether state adversaries or non state adversaries,
19:33
in developing chemical, biological, radiological, or nuclear weapons.
19:38
or cyber weapons, hacking, hacking tools and whatnot, and
19:41
automated attack approaches.
19:44
We will talk at more length about that because it ties in to the NTIA
19:48
process, but really the key issue there, as is the key issue generally,
19:53
is there actually a marginal risk?
19:55
Does open source pose a greater risk of those things than other technologies?
19:59
And I'm not going to spoil that.
20:02
I'll tell you how that turned out.
20:04
A third category is simply content issues of harmful or problematic
20:09
or illegal content, particularly of concern are deep faked child sexual
20:15
abuse material, virtual CSAM or deep faked NCII non consensual intimate
20:21
imagery, often called revenge porn.
20:24
And this is going to be a problem and is a problem.
20:27
But again, the marginal risk issue is a question that needs to be
20:30
asked, and we'll ask it shortly.
20:33
Finally, the fourth category, and where I think most of the concern
20:36
is settling now, and which I'm sure we'll talk about later, is China.
20:41
China specifically.
20:42
What do we do about Chinese competition in the AI field?
20:48
How do we deal with China as a national security competitor
20:53
in regard to AI as well?
20:55
And I think we are at a point now.
20:57
Where that's probably the most relevant consideration for policymakers in the U.
21:03
S.
21:04
As they look toward possible restrictions on open source now.
21:08
But, so that, that's the sort of range of risks I think
21:11
people are most concerned about.
21:14
Okay, let's drill down on some of those.
21:17
I, I agree we can put aside the existential risk.
21:19
But with regard to marginal risk, you made the point that, that guard
21:24
rails on these foundation models are fragile, but isn't it true that it's
21:30
substantially easier to remove a guard rail if you have access to the weight.
21:35
So if it's a, an open model, then it's fairly trivial for someone to take a,
21:40
that's a hostile actor to take out those guard rails, much more challenging.
21:44
If the company keeps control over the model.
21:46
It depends the you may have seen chat GTPs.
21:53
model released last week.
21:54
And then within minutes on Twitter, people sharing their, their attacks,
21:59
their prompts that were able to get around certain content guardrails.
22:04
The Taylor Swift sexually explicit deep fakes that we saw making the
22:08
rounds earlier in the summer, those were generated by Microsoft's.
22:13
Closed image creator, but yes, open source code, open source models are
22:20
more easily modifiable in a wide range of ways, including by bad actors.
22:26
But then that, that, that leads to the question of what about other technologies?
22:30
How does it compare to those?
22:31
And that's something that the NTIA got into, but to set up the NTIA
22:34
process a This came up in the context of the Biden AI executive order that
22:41
came out October 30th, I want to say last year, and which called on NTIA
22:48
to do a report on open weights AI.
22:53
This was prompted as best we can tell by there being staff at the National Security
22:59
Council of the White House and staff at the Bureau of Industry and Security,
23:02
the export control guys at the Commerce Department who were concerned particularly
23:07
about the CBRN and cyber risks.
23:09
And we're considering the possibility or looking at the possibility
23:16
of putting restrictions on the publication of open weights because
23:19
of those national security concerns.
23:21
It seems that somewhat cooler heads prevailed in pushing
23:25
instead pushing for that.
23:27
Not to be decided in the EO, but rather to delegate to NTIA, the National
23:31
Telecommunications Information and Information Administration, essentially
23:35
the president's lawyer, the president, not the president's lawyers, the
23:38
president's technology advisors, um, to do a more sustained study of the issue.
23:44
The issue in particular being what are the benefits of open weights models?
23:50
What are the risks of open weights models based on that?
23:53
What are potential policy approaches the president should take?
23:58
Or not and and so that was the process and they put out the call for comment
24:03
earlier this year A bunch of people filed comments including us and i'll
24:07
talk about that And then they finally released a report I guess a couple of
24:11
months ago And so let's talk about that and i'll talk about the comments that
24:15
cdt filed first a not so humble brag.
24:17
I'm proud to say This is a dubious achievement, but our comments were the
24:21
longest and most detailed comments filed in the proceeding, which had hundreds upon
24:25
hundreds of comments, more salient, our comments were cited in the report from
24:31
the NTIA more than any other comments.
24:34
And the only other single source that was cited more than our comments was a
24:39
paper that we coauthored with academics at Stanford and Princeton focused on
24:43
defining marginal risk around open models.
24:46
I'm glad to say, I feel that we had a very positive influence on this process.
24:51
We also worked closely with NTIA to ensure they heard from a variety of other civil
24:56
society voices from civil rights groups to civil liberties groups to open source
25:00
advocates and everything in between, but the gist of what we were saying in the
25:06
paper I mentioned in our comments is.
25:10
We are not ruling out categorically the possibility that some restrictions
25:14
on open weights may ultimately be necessary, but at this point at, with the
25:21
capabilities we see online now and that are coming shortly, there is not enough
25:27
evidence to support those restrictions.
25:31
And by, by that, not enough evidence that there's actually a significant
25:34
marginal risk from open models.
25:37
This is probably clearest.
25:40
In the context of the like biological and nuclear and cyber conversation there by
25:47
the time of our comments, there had been several academic papers and papers from
25:54
security think tanks like Rand research by Microsoft and open AI research where they
26:00
basically set up teams to compete on who could come up with the best evil terrorist
26:05
plan around biological weapons and gave One team access to the best models
26:12
and one team access to the internet.
26:15
And what they found was there was very little difference in the capacity of
26:20
those teams to plan an attack, which, which stands to reason because what the
26:26
models know is what they've seen online.
26:29
And so it may give some incremental additional aid in.
26:34
in collecting some of your sources when you're looking for information
26:37
to help plot a plot, but it really didn't make a huge difference.
26:42
Similarly, There was research on how much models, which certain models can also
26:49
help instead of just speaking in language, they can also code, they can create
26:53
software, and so there were tests about how effective they were in helping create
26:59
penetration tools and other attack tools.
27:02
And again, they found only a incremental difference, not really a major difference.
27:08
between what a coder could do without a model and what a coder could do with a
27:12
model in terms of offensive cyber stuff.
27:16
It's also worth noting, as with the case with open source software, that capability
27:23
to, to create tools also is now in the hands of defenders as well as attackers.
27:28
And as we've seen in the open source context, that has tended to ultimately
27:33
benefit defenders more than attackers.
27:35
And so NTIA, which was primarily focused on those as the most realistic,
27:40
National security related threats that they were asked to assess
27:44
essentially concluded We don't see a significant marginal risk here.
27:49
That doesn't mean there might not be one at some point.
27:53
Therefore, we do endorse continued monitoring.
27:58
I guess what a doctor would call watchful waiting, like it's not something that
28:01
requires intervention now, but something that we need to be watching closely,
28:05
including recommending continued investment in the development of best
28:11
practices and standards around safety.
28:13
As well as market surveillance and other measures to see how
28:18
these models are being used.
28:20
Because so much of the risk is actually going to turn on what context it is
28:23
being used and by what types of users.
28:26
And basically advise the president to hold off on restrictive measures.
28:31
And I think this was the right decision in a number of ways.
28:36
But especially by analogy to a similar fight over tech policy that we had
28:42
in the 90s, that we haven't talked about yet, but that has to be talked
28:45
about in this context, which is the fight over open source encryption
28:50
code in the 90s and early 2000s.
28:53
I think we have a very similar issue with AI today.
28:57
And this is now I'll talk about China, because it's a very similar situation
29:02
where in the encryption case, it was.
29:04
We are afraid our adversaries will misuse this technology and
29:08
therefore we want to try to limit the spread of this technology.
29:11
But the technology we're talking about is literally bits.
29:14
It's not a physical thing that we can physically constrain.
29:18
And so it's really hard to prevent the spread of software on a global internet.
29:23
And that's as true today, if not more than it was then.
29:27
And there are those that are concerned that if we are doing open source AI models
29:33
that We are giving away our IP to our competitors in China, that we are giving
29:39
away software that could be integrated into military operations by China.
29:46
Both of those are true in the sense that certainly there will be stakeholders in
29:50
China that make use of this software, but.
29:54
Even if our software is not available, they will make use of similar software,
29:58
whether it comes from France or the United Arab Emirates or China itself, where
30:04
there are plenty of large models being created today without reliance on U.
30:10
S.
30:10
technology.
30:11
So I think it's a question of, yes, we could attempt to restrict this.
30:16
We would probably fail.
30:18
It would probably leak anyway, but even if we did not fail, it
30:22
likely wouldn't be effective.
30:24
in preventing China from getting comparable technology elsewhere.
30:29
Meanwhile, we will have slowed down our own innovation cycle, slowed down the
30:36
diffusion of the technology generally, and the many economic benefits and
30:41
other benefits that will accrue from that, and have hindered our ability
30:46
to compete in the global marketplace.
30:49
Around this kind of software, which I think is especially important to consider
30:54
when you think about China's geopolitical position right now and things like
30:58
the Belt and Road Initiative, which is its massive global attempt to provide
31:03
infrastructure, including communications infrastructure to the global South,
31:08
to Africa, to South America, to the Middle East as a way of consolidating
31:13
geopolitical power, our having a robust open ecosystem of AI models.
31:19
Is probably our best shot at preventing China from dominating the
31:26
AI market in those global markets.
31:30
And they're actually at a unique disadvantage compared to our open
31:33
models because, because they are China.
31:37
The CCP has already passed regulations that basically enforce
31:42
ideological purity on Chinese models.
31:46
Basically, censorship in line with The party line in china, which makes
31:52
them necessarily much less useful to a lot of stakeholders So if we can
31:56
compete on a global stage With free, powerful, customizable AI software,
32:05
and China is competing with less good software, then that actually strengthens
32:10
our geopolitical position in regard to China, rather than weakening it.
32:15
Would be Our argument and the argument at this point of a number of relatively
32:21
conservative stakeholders as well as your center left types like me Okay,
32:26
there's a lot more that I would love to get into this with you about but
32:30
we have limited time I don't want to just ask one or two more questions.
32:34
Sure One is that the company used to work for, Meta, is the only one
32:40
of the major frontier AI labs that, that seems to be pushing forward with
32:44
releasing its most powerful models, the LLAMA models as open weights.
32:50
So can you understand, of course, you, you are not speaking for them.
32:54
Can you talk a little bit about.
32:56
Maybe why it seems like they've taken that direction and how is that going
33:01
to affect this debate that it's not now just an issue about AI safety, but it
33:05
plays into all of the competitive and other debates in the technology industry.
33:10
Sure.
33:10
Yeah, I'm not going to attempt to divine the mind of Mark
33:13
Zuckerberg or say anything.
33:16
Confidential that I may or may not have learned inside of Meta, but I
33:19
can note some things that other people have said and some obvious facts.
33:24
I think one of them is simple is the simple fact that Facebook now
33:30
Meta doesn't have a cloud services division through which it wants
33:34
to sell access to closed models.
33:37
It's it's whole business model is different.
33:40
It offers consumer products.
33:42
To consumers that rely on AI, but the them dominating the AI
33:49
model market is not their goal.
33:52
As one could argue is the goal of your Google's, your Microsoft's,
33:55
your Amazon's, your open AI's.
33:58
So they just have different incentives.
34:00
One of those incentives is by opening their largest model, which is also,
34:07
they've said the model that is going to be the primary basis of a lot of features
34:12
in their products, they can get the traditional benefits of open development.
34:17
They can get those millions of eyes on their, on their model.
34:21
They can benefit from seeing what customizations people
34:26
are doing on platforms like.
34:28
GitHub or HuggingFace and integrating the good ones into their production model.
34:33
And then there's also simply finally the obvious fact that it is what
34:38
they meta when ChachiTP dropped.
34:41
Meta had long been doing open source releases of research models.
34:46
Some may recall the dubious release of Galactica shortly before ChachiTP,
34:51
which was a, an LLM trained on and focused on scientific literature.
34:56
That spouted nonsense and was taken down pretty quickly.
34:59
And so you already had this large research investment in developing LLM technology
35:05
that wasn't necessarily being productized.
35:08
And there's the obvious fact that if.
35:10
Meta threw that out there while open AI and Microsoft were trying
35:14
to consolidate a closed AI position.
35:17
It would slow down those competitors.
35:19
Many of your listeners may have heard of the memo inside of Google.
35:23
I believe it was called open AI has no moat and neither do we in the sense of
35:27
a competitive moat against open source.
35:31
And I think.
35:32
People inside of Facebook probably followed a similar logic.
35:35
I am neither confirming or denying.
35:37
And in fact, I was on parental leave for much of this decision making, but
35:40
recognized that offering this would not only create a lot of benefit and use
35:46
for a lot of people, but also slow the adoption of closed models offered by
35:51
their competitors, who again, Meta doesn't offer a cloud service, but it does offer
35:57
a lot of other services that are in competition with a lot of these companies.
36:00
And so I think as a number of commentators have noted, that was a
36:04
clear benefit to Facebook if they could throw an obstacle in front of their
36:09
fast moving opponents in the market.
36:12
As I make the distinction between open weights and open source, Meta is
36:16
a good example of why we need to be really careful about our terminology
36:19
there, because Mark Zuckerberg loves to call LLAMA open source and use
36:24
that terminology, but the license that Meta is licensing under is
36:29
definitely not an open source license.
36:32
In some ways that are arguably good and in some ways that are arguably bad.
36:36
Most notably, their original license forbade you from building
36:41
a model using outputs from their model, which is actually a very
36:44
powerful way of building smaller, more condensed versions of models.
36:48
Thankfully, they fixed that.
36:49
But now if you do that, your model has to be Marked as built with llama and you
36:58
don't typically see that kind of branding requirement in the open source community
37:03
And so I'd love for them to get rid of that and then also as has been much
37:07
commented on they also have a clause that prohibits any company with more than 700
37:13
million average monthly users To use the model this is to basically prevent google
37:19
and microsoft from using the model for free And if they want to use it, they'll
37:23
have to go to to meta to license it again Like that's not the sort of thing you'd
37:28
see in a regular open source license carving out particular users At least
37:33
it is limited to Companies that had that level of users at the time of the license.
37:38
So it's not like you could build your company on LLAMA, ultimately get to 750
37:43
and then suddenly have the bottom pulled out of your business, but it's still fair
37:48
to say not anti competitive and something that, that especially if they want to
37:53
keep calling it open source, they need to take out of their license or they
37:57
need to stop calling it open source.
38:00
All right, there's a lot more that we could talk about here Obviously the
38:03
ntia report will not be the last word, but we're going to need to wrap up for
38:07
time kevin Thank you so much for going through and in so much detail this really
38:11
important set of issues with us Anytime.
38:13
It was a pleasure.
38:15
Great to have you.
38:17
This has been The Road to Accountable AI.
38:19
If you like what you're hearing, please give us a good review and
38:22
check out my sub stack for more insights on AI accountability.
38:26
Thank you for listening.
38:35
This is Kevin Werbeck.
38:36
If you want to go deeper, check On AI governance, trust, and responsibility
38:39
with me and other distinguished faculty of the world's top business
38:43
school, sign up for the next cohort of Wharton's strategies for accountable
38:47
AI online executive education program, featuring live interaction
38:51
with faculty, expert interviews, and custom designed asynchronous content.
38:56
Join fellow business leaders to learn valuable skills you can
38:59
put to work in your organization.
39:01
Visit execed.
39:03
wharton.
39:03
upenn.
39:04
edu slash ACAI for full details.
39:08
I hope to see you there.