Latest / Elon Musk Podcast / Elon Musk fights California over First Amendment

Transcript

Elon Musk fights California over First Amendment · Elon Musk Podcast

0:00
Artificial intelligence developers are running out of
0:02
high quality human data to train their systems, and it's forcing
0:06
them to train new AI on text generated by older AI, which is
0:11
a process that eventually causes the models to completely
0:13
collapse and start spitting out gibberish about jackrabbits when
0:17
you ask them to describe historical English architecture.
0:19
Right the the jackrabbit hallucination it.
0:22
It's just such a bizarre, striking visual to start with
0:25
it. Really is because.
0:26
You have these systems, right? They're engineered to be the
0:29
most sophisticated heated synthesis of human knowledge
0:31
ever assembled. They ingest entire libraries,
0:35
thousands of years of recorded human thought, and they process
0:38
it beautifully. But the moment they run out of
0:41
our writing and they start consuming their own recycled
0:44
data. So when an AI trains on the
0:45
output of another AI, the internal logic just completely
0:49
deteriorates. I mean, it's a concept engineers
0:51
call synthetic data inbreeding, which sounds gross, but it's
0:54
accurate. Yeah, over successive
0:56
generations of training, that mathematical anchor to human
0:59
reality just begins to drift by the 3rd or 4th generation of it
1:03
basically eating its own tail. A model asked to discuss the
1:07
structural elements of a 14th century cathedral will suddenly
1:10
start generating paragraphs about the breeding habits of
1:13
Jack rabbits. The whole system just eats
1:15
itself and collapses into absolute nonsense.
1:17
Exactly. And that bizarre reality sets up
1:20
the core tension we're looking at today.
1:23
You have massive technology companies actively hiding the
1:26
exact ingredients of their models to protect these
1:30
incredibly lucrative empires. Because high quality human data
1:34
is suddenly the most scarce, valuable commodity on Earth.
1:37
Right. And meanwhile, new laws are
1:39
attempting to RIP the lid off that exact same black box to
1:42
expose exactly whose data is being consumed.
1:45
Which brings us to the real issue here.
1:47
Can lawmakers actually force technology companies to expose
1:50
the secret ingredients of their artificial intelligence without
1:53
destroying the value of the models themselves?
1:56
I mean, the sheer scale of the legal requirements now hitting
1:59
these companies is staggering. New regulations require any
2:03
developer releasing A generative AI system to the public to post
2:07
a high level summary of their training data sets right on
2:10
their website. Which seems huge.
2:12
It completely shatters the old standard of simply releasing a
2:15
product and telling you not to worry about how it got so smart.
2:18
Right, but a high level summary sounds simple enough on the
2:21
surface. Just give us a basic overview.
2:23
But reading through the actual documentation required by these
2:27
mandates, it is incredibly demanding.
2:29
Yeah, the wording is definitely deceptive.
2:32
The regulations dictate that companies must disclose the
2:35
origins of the data and the specific sources or owners of
2:38
those data sets. Which was a massive undertaking.
2:40
And they have to provide the exact number of data points
2:42
included in the sets. That alone is an astonishing
2:45
requirement when you consider the architecture of these
2:47
systems. We are not talking about
2:49
counting books on a shelf here. We are talking about models that
2:52
process billions, sometimes trillions, of individual tokens
2:56
of information. They also have to disclose
2:59
whether the data is copyrighted or in the public domain, whether
3:02
it was purchased or licensed. Back up.
3:05
Yeah. What exactly qualifies as a high
3:07
level summary? Because listing out trillions of
3:10
individual data points, categorizing them by copyright
3:13
status and tracking down the original owners sounds like the
3:16
exact opposite of a summary. Yeah.
3:17
And just to ground that for you listening, a token isn't a whole
3:20
word, right? It's more like a syllable or a
3:22
fragment of a word. So trillions of tokens is an
3:26
unimaginably granular breakdown of human language.
3:29
Exactly. A token is essentially the
3:31
smallest unit of meaning the machine can process.
3:34
Sometimes it's a word, sometimes it's a prefix like on or pre,
3:38
sometimes it is just a single character.
3:40
Imagine being asked to provide a summary of a library, but the
3:43
law requires you to classify every single syllable in every
3:47
single book. And note where that syllable was
3:49
printed. Right.
3:50
And who holds the rights to the paper it is printed on and
3:53
whether the ink used was synthetic?
3:55
This mandate fundamentally limits the historic culture of
3:58
technology development. How so?
4:01
Well, for years the standard operating procedure was to write
4:04
an automated script, scrape the entire public Internet as
4:08
quickly as possible, feed it into a massive server farm, and
4:12
just see what the neural network could learn.
4:15
Nobody was meticulously cataloguing the input.
4:17
They were just vacuuming it all up.
4:18
Exactly. Now developers are forced to
4:21
conduct exhaustive microscopic audits of all data collection
4:25
practices retroactively. It changes how they categorize
4:29
and store information before a single line of code is ever
4:32
executed to build the model. If you have to publicly declare
4:35
every single category of data, you can no longer afford to be
4:39
messy with your ingestion pipelines.
4:41
And if we look at the potential fallout of doing that
4:43
accurately, combining these individual disclosures could
4:46
theoretically allow competitors to reverse engineer the exact
4:49
data recipes used by the industry leaders.
4:51
Oh. Absolutely.
4:52
Because if I know exactly what percentage of your training data
4:55
came from specific types of copyrighted text, exactly how
4:59
much synthetic data you mixed in to smooth out the edges, and
5:02
exactly what proportion of conversational data you
5:05
prioritize, I have a massive head start.
5:08
You get the map without having to explore the wilderness
5:11
yourself. Right.
5:12
I can replicate your success without spending the billions of
5:15
dollars you spent on trial and error.
5:17
Which brings us to how the major players are actually responding
5:20
to these regulatory mandates. Major companies, including
5:23
Anthropic and Open AI are complying with the disclosure
5:26
laws by using incredibly vague language.
5:29
Of course they are. They are checking the required
5:31
boxes by citing broad nebulous categories like publicly
5:35
available information or non public data from third parties.
5:39
They are actively avoiding naming a single specific data
5:42
set or revealing the exact ratios of their mixtures.
5:46
They're essentially saying yes, we use data and yes, it came
5:49
from the Internet. It feels like compliance through
5:51
malicious compliance. They are giving the regulators
5:55
words without giving them any actual meaning.
5:58
And then you have Elon Musk's XAI, which took a much more
6:02
aggressive confrontational route.
6:04
They attempted to secure a preliminary injunction to block
6:07
these transparency laws entirely.
6:09
Yeah, their argument in court was that the law forces them to
6:12
surrender proprietary secrets, though a federal judge outright
6:15
denied the request. Right.
6:17
But from a strictly business and legal perspective, the logic
6:21
behind attempting to block the law does make sense.
6:24
Forcing companies to reveal their exact weights, parameters
6:27
and training mixtures is essentially equivalent to an
6:29
unconstitutional taking of private property and trade
6:32
secrets. Do you think so?
6:34
Well, think about it. The specific blend of data, the
6:36
exact filtering mechanisms used to clean that data, the precise
6:40
ratio of synthetic to human text.
6:42
That is the intellectual property.
6:44
That is the secret sauce that makes one company's model
6:47
hallucinate less than another's. Forcing them to publish that
6:50
recipe hands their hard earned competitive advantage over to
6:53
anyone with a high speed Internet connection and a
6:55
cluster of graphics processors. OK, I have to push back there.
6:58
Go ahead. You cannot call it an
7:00
unconstitutional taking of property when the entire system
7:04
is quite literally built on the uncompensated property of
7:07
millions of creators. You can't claim your secret
7:10
recipe is a protected trade secret when the ingredients were
7:14
scraped from personal blogs, news sites and digital
7:16
portfolios without permission. That is the counter argument,
7:19
yes. Corporate secrecy cannot
7:21
function as an impenetrable shield, especially when state
7:25
attorneys general are actively investigating the outputs of
7:27
chat bots like Grok. When these systems are
7:30
generating content that impacts elections, offering questionable
7:34
medical advice, or mimicking the voices and likenesses of real
7:37
people, that companies cannot just stand behind closed doors
7:40
and say, trust us, our recipes a trade secret, right?
7:44
The public and the lawmakers representing them have a right
7:47
to know what is feeding the machine that is suddenly making
7:50
decisions for them. And the companies hear that
7:53
argument loud and clear. And that legal friction changes
7:56
their entire compliance strategy.
7:58
What are they doing instead? It opens up a totally new
8:01
corporate approach, which is the of trust centers.
8:05
Instead of listing raw data points and handing over the
8:07
exact ingredients to appease the lawmakers, companies are
8:11
publishing extensive, beautifully designed
8:13
documentation focused entirely on their privacy preserving
8:16
filters and their safety techniques.
8:19
Redirection. Completely.
8:20
They are building highly polished web portals dedicated
8:23
to explaining how much they care about security, how many
8:27
automated red teams they deploy, and how rigorously they test for
8:30
bias. So it looks like transparency.
8:32
It builds consumer confidence. It satisfies the surface level
8:36
requirements of showing that they have governance structures
8:39
in place while keeping the actual valuable raw data
8:42
completely hidden from view. So they tell you exactly how
8:45
strong the locks on the doors are.
8:47
They show you the security cameras, they introduce you to
8:49
the guards, but they never actually show you what is inside
8:51
the vault. That is a perfect way to
8:53
describe it. Let's talk about what happens
8:55
when your personal diary entry or your private e-mail
8:59
accidentally gets locked inside that vault with no way to get it
9:03
out, because this leads directly to a massive collision with
9:07
consumer privacy. Consumer privacy laws are being
9:10
officially extended to cover artificial intelligence systems.
9:13
Yes, that means personal data cannot be mined or regurgitated
9:17
without accountability. And the massive challenge here
9:20
is the very nature of the automated collection process.
9:24
Personal information things like private e-mail addresses, cell
9:27
phone numbers, or even highly sensitive health records
9:30
discussed on obscure support forms a decade ago is frequently
9:34
swept up as a purely incidental byproduct of massive Web
9:37
crawling. They aren't looking for it
9:39
specifically. Right.
9:40
The developers building these systems are not actively
9:42
searching for your specific personal e-mail address to teach
9:45
their machine how to write, but when they deploy A crawler to
9:48
vacuum up an entire domain to learn the structure of human
9:51
conversation, your data comes along for the ride.
9:53
Hold on, if the data is swept up accidentally, how do they remove
9:58
it once the AI has already learned it?
10:00
Because from everything in these technical briefs and AI does not
10:04
store data like a standard computer or hard drive where you
10:07
can just find a file, highlight it and hit delete.
10:10
It does not work like a database at all.
10:12
And that engineering reality changes the entire architecture
10:16
of AI development because. You can't just run a search
10:18
query. Exactly.
10:19
In a normal database, if you request your data be deleted
10:22
under privacy laws, a company runs a query, locates your row,
10:26
and erases it. A neural network stores
10:29
information in weights and connections across billions of
10:32
parameters. If a user exercises their legal
10:35
right to request their personal information be scrubbed,
10:37
developers face the nearly impossible task of altering the
10:40
foundational architecture of an already trained model.
10:43
So how do you remove something that's baked in?
10:45
Think of it like trying to remove a single drop of red dye
10:48
from a fully baked cake. The dye isn't sitting in one
10:51
spot, it has structurally altered the entire composition
10:55
of the cake. This reality severely limits the
10:58
indiscriminate vacuuming of the Internet that fueled the first
11:01
wave of these tools. They have to be careful from the
11:03
start now. And it opens up the absolute
11:05
necessity for post training techniques like implementing
11:08
specific guardrails, automated filters and a concept called
11:12
machine unlearning to minimize personal information from ever
11:16
appearing in models outputs, even if the model technically
11:20
retains the mathematical memory of that data deep within its
11:23
neural pathways. That is a terrifying thought.
11:26
If your data gets vacuumed up, it is permanently baked into the
11:29
model's brain. There is no actual delete
11:32
button, just a muzzle put on the machine so it hopefully doesn't
11:34
repeat what it knows about you. A muzzle is a good way to put
11:37
it. And it is not just personal
11:39
information getting caught in that indiscriminate web crawler.
11:43
Tech giants deliberately transcribed copyrighted Internet
11:47
videos and gathered protected text across the Web because
11:51
negotiating licenses with individual publishers,
11:53
independent artists and musicians would simply take too
11:56
much time. It was a race.
11:58
Internal communications from several companies revealed they
12:01
knew exactly what they were doing, calculating that the risk
12:04
of facing future lawsuits was entirely worth the immediate
12:08
reward of building a smarter model faster than their
12:10
competitors. And the proposed transparency
12:13
measures take direct aim at that specific practice.
12:16
Under new regulatory frameworks, developers would be required to
12:19
respond to copyright owners with a complete, verified list of
12:23
materials used. Which goes back to the indexing
12:25
problem. Yes, and they would achieve this
12:27
using an approximate content fingerprint to search for
12:30
matches in their massive data sets.
12:32
Essentially, a creator provides a digital signature of their
12:35
work. Think of it like an audio
12:37
identification app on your phone that can listen to a song in a
12:40
noisy room and identify the underlying melody even if the
12:44
instruments sound different. Although it makes sense.
12:46
You create that same kind of signature for a textbook or a
12:49
piece of code, and the AI company has to run that
12:52
mathematical signature against their entire training library to
12:55
see if there is a match. This proposed tracking mechanism
12:58
completely changes the balance of power.
13:01
Theoretically, it gives creators the exact ammunition they need
13:05
to sue for copyright infringement or demand financial
13:08
compensation. Yeah, if you can mathematically
13:11
prove your fingerprint is in their system, you have a solid
13:14
legal case. Yes, but there's a downside.
13:16
Right. The massive downside is that it
13:18
limits the speed of innovation. Verifying millions of individual
13:22
fingerprints against massive data sets containing trillions
13:26
of tokens is computationally heavy.
13:28
It requires an enormous amount of processing power just to run
13:32
the compliance checks. You are essentially asking a
13:34
supercomputer to search for millions of needles in a
13:37
haystack the size of a planet continuously every time a new
13:41
creator registers a fingerprint. So the companies are being
13:44
squeezed from both sides. They require infinite pristine
13:47
data to make the model smarter and avoid the model collapse we
13:50
talked about earlier. The Jack rabbits.
13:52
Exactly. But every single piece of data
13:55
they ingest now carries a legal tripwire that could result in
13:59
massive financial penalties or, worse, forced architectural
14:03
changes to the models themselves.
14:05
If you've got a decent mic and laptop and some free time,
14:08
Babble Audio is paying people to record speech data and annotate
14:11
audio for AI training. No minimums, no fixed hours.
14:15
You work when you want and get paid weekly via PayPal, Venmo,
14:18
or bank transfer. They pay per recorded or
14:20
annotated hour, plus bonus challenges for hitting weekly
14:23
goals. And if you sign up through our
14:25
link, you get priority processing and a $15 bonus.
14:28
Links in the show notes. So getting back to it, beyond
14:30
the massive copyright issues and the privacy nightmares, there is
14:33
an entirely different category of regulation targeting the
14:36
absolute biggest players in the room.
14:38
Developers of massive frontier models, meaning those generating
14:41
massive annual revenue and utilizing raising immense
14:44
computing power for training, are now legally required to
14:47
report critical safety incidents directly to state emergency
14:50
services. Which is wild to think about for
14:52
a software company. The internal governance
14:54
requirements placed on these frontier developers are
14:57
incredibly strict and completely unprecedented in the software
15:00
industry. They must provide anonymous
15:02
reporting hotline specifically for their engineering staff.
15:06
They have to update those whistleblowers monthly on the
15:08
status of any internal investigations, and they are
15:11
required by law to report those findings directly to their board
15:15
of directors. The legal framework completely
15:18
bypasses the middle management layers that typically bury these
15:21
types of concerns to keep product launches on schedule.
15:24
Let me pause you right there 'cause I want to make sure I am
15:27
wrapping my head around this. What kind of emergency are we
15:29
talking about here? When I think of a chat bot, even
15:32
a highly advanced one, I do not typically think of someone
15:35
needing to dial emergency services because the software
15:38
produce a bad output. No, you wouldn't normally, but
15:41
this is specifically targeted at catastrophic risk to public
15:45
health or physical safety. The regulators are looking at
15:48
the trajectory of the technology.
15:50
We are talking about models becoming capable of generating
15:54
novel, highly lethal chemical weapons formulas that bypass
15:58
known security filters. Yeah, we are talking about
16:03
models tasked with coding that independently find and exploit 0
16:07
day vulnerabilities in critical infrastructure grids like water
16:11
treatment plants or power stations, or models given
16:14
autonomous agent capabilities that engage in actions that
16:17
could cause severe physical harm or massive financial disruption.
16:21
So real world consequences, not just digital ones.
16:24
Exactly. This mandate completely changes
16:27
the accountability structure for corporate executives.
16:29
It limits their ability to hide dangerous flaws or terrifying
16:33
emergent capabilities in their technology to protect their
16:36
public stock price. But.
16:37
They have to report it. Right.
16:39
It opens up an environment where AI employees have protected
16:42
legally two mandated channels to sound the alarm if a model
16:46
begins behaving dangerously during the training run, and it
16:49
forces the company to engage with state and Federal Emergency
16:52
services if those alarms are determined to be valid.
16:55
That is a massive shift from the standard software development
16:58
life cycle. Usually if software has a bug,
17:01
you patch it. You don't call the authorities.
17:03
But there is a massive catch to all of this.
17:06
Force transparency and reporting.
17:08
Academic researchers warned that simply forcing AI companies to
17:12
dump massive amounts of data documentation on the public is a
17:15
deeply flawed strategy. They compare it to the endless
17:18
privacy policies or complex Federal Home Loan disclosures
17:23
that consumers click agree to but never actually read.
17:26
It's the fine print problem. Yes, it reminds me of those 50
17:29
page terms of service updates we all blindly accept on our
17:32
phones. Except now it is for a machine
17:35
that might be writing our medical prescriptions or
17:37
filtering our job applications. The psychological reality is
17:40
that overwhelming the public with highly technical data
17:43
points creates intense information overload.
17:46
When a company complies with a mandate by publishing A500 page
17:49
technical document detailing their data set categorization,
17:53
their parameter counts, and their safety benchmarks, it
17:55
actually reduces genuine transparency for the average
17:58
person. Because nobody can read it.
17:59
You hide the needle in a massive haystack of compliance jargon.
18:03
If I hand you a textbook on fluid dynamics, I haven't
18:05
actually helped you understand how your plumbing works.
18:07
That's a great analogy. And this realization severely
18:10
limits the effectiveness of raw data dumps as a tool for public
18:14
accountability. It changes how policy makers
18:17
must approach regulation entirely.
18:19
Because direct to consumer doesn't work here.
18:21
Right. Instead of focusing on direct to
18:23
consumer disclosures that no one will ever read or understand,
18:27
they are opening up a framework designed specifically for
18:30
information intermediaries. These intermediaries are the
18:33
investigative journalists, the academic researchers, and the
18:36
independent nonprofit watchdogs. The translators, essentially.
18:40
Exactly. They are the ones equipped with
18:42
the technical knowledge to analyze the raw data, run the
18:45
necessary adversarial tests against the models, and
18:48
translate those dense, unreadable disclosures into
18:51
actionable, understandable insights for the general public.
18:54
And those intermediaries are going to need every advanced
18:57
tool they can get their hands on, because the technical
18:59
reality of catching these models breaking the rules is becoming
19:03
incredibly difficult. Legal audits that attempt to
19:06
catch AI companies stealing data by searching for exact word for
19:10
word matches, which are known in computer science as substring
19:14
searches, are becoming entirely obsolete.
19:17
So the AI can steal your work, rewrite it just enough to pass a
19:21
standard plagiarism check, and the law cannot prove it took
19:25
your specific data. That is the exact problem the
19:27
legal system is facing right now.
19:29
Modern AI models do not operate like a search engine.
19:32
They do not simply memorize and paste text from a database.
19:36
They synthesize and regurgitate concepts by stitching together
19:39
mathematical fragments. So they learned the idea, not
19:41
just the words. The model maps language in a
19:44
multi dimensional space. Words that have similar meanings
19:47
are grouped closer together in that space.
19:49
A model can perfectly replicate a highly specific copyrighted
19:53
concept or a unique fictional narrative without ever using the
19:56
exact original sequences of words.
19:59
Because it understands the meaning.
20:00
Right. The neural network understands
20:02
the semantic meaning of the text, so it can express the
20:04
exact same idea using an entirely different vocabulary
20:08
and sentence structure. If your legal audit relies on
20:11
finding identical strings of words looking for a copied
20:14
paragraph, you will find absolutely nothing.
20:16
The audit will come back clean even if the model was entirely
20:20
trained on your specific protected book or article.
20:23
So essentially trying to catch these models stealing data using
20:27
old software tools cause like trying to nail Jelly to a wall,
20:31
the law is trying to catch a ghost using a net designed for
20:35
physical objects. It just phases right through.
20:37
This technological reality severely limits the enforcement
20:40
of both data transparency and copyright laws.
20:43
It changes the technical requirements for future audits,
20:46
opening up a massive demand for new advanced solutions like
20:49
probabilistic testing. Which is fascinating.
20:52
Yeah, that involves asking the model a highly specific question
20:54
millions of times to statistically prove it knows
20:57
something it shouldn't. It also opens up the demand for
21:00
persistent data watermarks. Right, the invisible patterns.
21:03
Invisible statistical patterns embedded into the original
21:06
content itself that alter the Model S weights during training,
21:10
creating a hidden signature that cannot be easily scrubbed or
21:13
altered by the AI company. Which leaves us looking at an
21:16
incredibly complex regulatory environment.
21:19
The speed of the technology, the sheer scale of the mathematics
21:22
involved, is actively outpacing the structural capacity of the
21:26
legal system to measure it, let alone control it.
21:29
Regulators are trying to force open the black box of artificial
21:32
intelligence with mandatory disclosures and whistleblower
21:35
protections. But the sheer complexity of how
21:38
these models learn, and how they obscure their own origins is
21:42
making true accountability a constantly moving target.
21:45
What happens when the underlying technology becomes so advanced
21:49
and the synthetic data loops become so dense that even the
21:52
developers themselves can no longer trace the true origin of
21:55
the machine's outputs? If you're not subscribed yet,
21:58
take a second and hit follow on whatever app you're using.
22:00
It helps us keep making this. We appreciate you being here.