Latest / Artificial Intelligence Exposed / Understanding Bias and Fairness in Automated Systems

Transcript

Understanding Bias and Fairness in Automated Systems · Artificial Intelligence Exposed

• 0:00
  Speaker 1: I want you to imagine a scenario for a second.
• 0:03
  Speaker 1: Just put yourself in this headspace. You have just applied
• 0:07
  Speaker 1: for your absolute dream job. Or, you know, maybe you're
• 0:10
  Speaker 1: trying to get a mortgage for your first house. Right.
• 0:12
  Speaker 2: Something that's really high stakes.
• 0:14
  Speaker 1: Exactly. High stakes. Or maybe you are sitting by the
• 0:18
  Speaker 1: phone waiting on the results of a really critical medical screening.
• 0:22
  Speaker 2: Oh, that's the worst feeling.
• 0:24
  Speaker 1: It really is. You wait.
• 0:25
  Speaker 2: You worry.
• 0:26
  Speaker 1: You refresh your inbox like a hundred times. And finally,
• 0:30
  Speaker 1: the response comes back. You've been rejected or denied. And
• 0:35
  Speaker 1: when you call up the company or the bank or
• 0:37
  Speaker 1: the clinic to ask why, when you desperately look for
• 0:41
  Speaker 1: a reason, the only explanation you get from the person
• 0:43
  Speaker 1: on the other end of the line is the algorithm
• 0:45
  Speaker 1: made the decision.
• 0:46
  Speaker 2: Wow. Yeah, it was a phrase that carries this, I
• 0:49
  Speaker 2: don't know, this terrifying ring of finality to it.
• 0:52
  Speaker 1: It really shuts you down.
• 0:53
  Speaker 2: It shuts down the conversation entirely. There's no arguing with it.
• 0:55
  Speaker 1: It really does. It's presented as this ultimate unarguable truth. Like,
• 1:00
  Speaker 1: it is math, so it just must be right.
• 1:02
  Speaker 2: Yeah, we're totally conditioned to think that way.
• 1:04
  Speaker 1: Right. But today, we are going into a deep dive
• 1:08
  Speaker 1: on this incredibly extensive eye-opening text. It's called Understanding Bias
• 1:13
  Speaker 1: and Fairness in Automated Systems.
• 1:16
  Speaker 2: It's a fantastic source. Yeah. Really comprehensive. It is.
• 1:19
  Speaker 1: And our core mission today for this deep dive is
• 1:22
  Speaker 1: to completely dismantle the myth of the objective machine.
• 1:26
  Speaker 2: Yes, that is exactly what we need to do.
• 1:29
  Speaker 1: Because the goal here is to equip you, the listener,
• 1:32
  Speaker 1: with the algorithmic literacy you actually need, you know, to
• 1:36
  Speaker 1: navigate a world that is just increasingly being run by
• 1:39
  Speaker 1: these automated decisions.
• 1:41
  Speaker 2: Which is, frankly, rapidly becoming a non-negotiable life skill.
• 1:44
  Speaker 1: Totally.
• 1:44
  Speaker 2: I mean, we have this deep-seated psychological tendency to view
• 1:48
  Speaker 2: these systems as infallible simply because they run on code.
• 1:51
  Speaker 1: Like it's magic or something.
• 1:52
  Speaker 2: Exactly. We equate computation with fairness. We think, well, it's
• 1:55
  Speaker 2: a computer, so it can't be biased.
• 1:57
  Speaker 1: Okay, let's unpack this because the text sets up this
• 2:00
  Speaker 1: really incredible initial contrast.
• 2:02
  Speaker 2: It does.
• 2:02
  Speaker 1: Think about it. Unlike a human being, a machine does
• 2:05
  Speaker 1: not get angry.
• 2:06
  Speaker 2: No, it doesn't.
• 2:07
  Speaker 1: It doesn't get tired. It doesn't feel jealous of a coworker.
• 2:10
  Speaker 1: And it certainly doesn't get impatient at the end of
• 2:13
  Speaker 1: like a 14-hour shift.
• 2:14
  Speaker 2: Right. It doesn't need coffee to function.
• 2:16
  Speaker 1: Exactly. So naturally, we assume that replacing our flawed, highly
• 2:22
  Speaker 1: emotional human judgment with a pristine, cold algorithm automatically guarantees fairness.
• 2:29
  Speaker 2: It sounds so logical when you put it like that.
• 2:31
  Speaker 1: It does. But the text makes it incredibly clear that
• 2:34
  Speaker 1: this assumption is just a dangerous illusion.
• 2:37
  Speaker 2: It's a very dangerous illusion because— It fundamentally misunderstands what
• 2:40
  Speaker 2: artificial intelligence actually is. Right. And, well, more importantly, it
• 2:45
  Speaker 2: misunderstands how it is built. AI does not learn from
• 2:48
  Speaker 2: some perfect neutral vacuum of pure logic.
• 2:51
  Speaker 1: It's not just floating in space. No.
• 2:53
  Speaker 2: It learns from data. And that data is produced by
• 2:56
  Speaker 2: human societies. It's produced by human institutions.
• 2:59
  Speaker 1: It's made by us.
• 3:00
  Speaker 2: Exactly. And it's built on centuries of highly flawed historical decisions.
• 3:05
  Speaker 2: The AI is essentially a mirror.
• 3:08
  Speaker 1: A mirror. I love that analogy.
• 3:09
  Speaker 2: Yeah, it is entirely capable of reproducing, amplifying, and even
• 3:15
  Speaker 2: just bluntly revealing all of our hidden structural inequalities.
• 3:20
  Speaker 1: So it's less like an all-knowing omniscient oracle and more
• 3:24
  Speaker 1: like a really attentive parrot.
• 3:27
  Speaker 2: A parrot. That's a great way to put it.
• 3:28
  Speaker 1: Yeah, like a parrot that has been listening to absolutely
• 3:31
  Speaker 1: everything we've been doing for decades, both the good stuff
• 3:33
  Speaker 1: and the very, very bad stuff.
• 3:35
  Speaker 2: It repeats what it hears.
• 3:37
  Speaker 1: Right. Let's dig into this idea of the historical mirror
• 3:41
  Speaker 1: because the text offers this brilliant example regarding a hiring algorithm.
• 3:45
  Speaker 2: Yes. The resume screening one.
• 3:47
  Speaker 1: Yeah. So imagine a massive corporation. Thousands of resumes are
• 3:51
  Speaker 1: pouring in every single month.
• 3:53
  Speaker 2: Just a mountain of paper.
• 3:54
  Speaker 1: Exactly. The HR department is drowning. They literally cannot read
• 3:58
  Speaker 1: them all. So they decide to automate the initial screening process.
• 4:01
  Speaker 2: Which makes sense on paper.
• 4:03
  Speaker 1: Right. They train a machine learning model using the data
• 4:05
  Speaker 1: of their past employees who received the strongest performance reviews.
• 4:09
  Speaker 2: Now, if you are the manager of that department, that
• 4:11
  Speaker 2: sounds like the most rational business decision in the world.
• 4:14
  Speaker 1: You just want to replicate success.
• 4:16
  Speaker 2: Exactly. You are essentially telling the mathematical system, hey, look
• 4:20
  Speaker 2: at the traits of our top performers and just find
• 4:23
  Speaker 2: me more people who look like them on paper.
• 4:25
  Speaker 1: Just copy and paste the good employees.
• 4:27
  Speaker 2: Right. But we need to look at why this goes
• 4:30
  Speaker 2: catastrophically wrong. Data is, like we said, a crystallized product
• 4:35
  Speaker 2: of history.
• 4:36
  Speaker 1: Okay, so how does the history ruin it here?
• 4:39
  Speaker 2: Well, suppose this specific company, historically, had a workforce that
• 4:43
  Speaker 2: was heavily concentrated in specific demographic groups.
• 4:46
  Speaker 1: Like mostly guys from certain backgrounds?
• 4:48
  Speaker 2: Exactly. Maybe their past recruiters, whether consciously or unconsciously, heavily
• 4:54
  Speaker 2: favored graduates from like two or three elite universities.
• 4:57
  Speaker 1: Oh, I see.
• 4:58
  Speaker 2: Or perhaps they favored people from a specific geographic region.
• 5:02
  Speaker 2: or even those who participated in very specific extracurricular social networks.
• 5:07
  Speaker 1: So basically the classic old boys club effect, or just
• 5:10
  Speaker 1: hiring people who happen to play golf with the CEO.
• 5:13
  Speaker 2: Right, or even just geographical convenience, like maybe the company
• 5:16
  Speaker 2: used to only recruit from the college down the street
• 5:18
  Speaker 2: because it was easy.
• 5:19
  Speaker 1: Makes sense.
• 5:19
  Speaker 2: But when the algorithm starts processing that historical data, it
• 5:24
  Speaker 2: isn't explicitly told to discriminate. No engineer is sitting there
• 5:28
  Speaker 2: writing a line of code that says, exclude applicants from
• 5:32
  Speaker 2: outside this demographic.
• 5:33
  Speaker 1: Right. They aren't trying to be evil.
• 5:35
  Speaker 2: Not at all. But the algorithm is a pattern matching engine.
• 5:40
  Speaker 2: It is just looking for statistical correlations of past success.
• 5:45
  Speaker 1: And past success looks like that old boys club.
• 5:47
  Speaker 2: Exactly. If past success in that company was deeply intertwined
• 5:51
  Speaker 2: with those specific universities or those regions, the algorithm will
• 5:55
  Speaker 2: quietly learn those indirect patterns.
• 5:57
  Speaker 1: And it just applies them to everyone.
• 5:59
  Speaker 2: It enforces them rigorously on all the new applicants.
• 6:02
  Speaker 1: You know, it makes me think of learning how to drive.
• 6:05
  Speaker 2: How so?
• 6:05
  Speaker 1: Well, imagine you're trying to learn how to drive by
• 6:08
  Speaker 1: exclusively watching someone who constantly rolls through stop signs.
• 6:12
  Speaker 2: Oh, right.
• 6:13
  Speaker 1: And, like, they never use their turn signal. If you
• 6:15
  Speaker 1: learn entirely from their historical driving data, you're going to
• 6:18
  Speaker 1: encode all of their bad habits into your own driving.
• 6:21
  Speaker 2: You wouldn't even know what's wrong.
• 6:22
  Speaker 1: Exactly. You won't know it's a traffic violation. You'll just think, well,
• 6:25
  Speaker 1: this is fundamentally how driving works. If you learn from
• 6:28
  Speaker 1: historical data, you encode historical biases.
• 6:32
  Speaker 2: That analogy holds up perfectly. The algorithm might become extremely
• 6:37
  Speaker 2: perfectly accurate at reproducing the past.
• 6:39
  Speaker 1: But we don't want the past.
• 6:41
  Speaker 2: Right. In a modern hiring context, your objective shouldn't be
• 6:44
  Speaker 2: to blindly reproduce your past. It should be to build
• 6:47
  Speaker 2: the best possible future for your company.
• 6:50
  Speaker 1: So if your historical decisions were unequal.
• 6:53
  Speaker 2: Or narrow-minded. Right.
• 6:54
  Speaker 1: Or narrow-minded. Training a machine to imitate those decisions perfectly
• 6:59
  Speaker 1: just preserves that inequality.
• 7:01
  Speaker 2: And worse, now it's hidden under this shiny, unquestionable veneer
• 7:05
  Speaker 2: of automation.
• 7:06
  Speaker 1: Because it's math. So it must be fair.
• 7:08
  Speaker 2: Exactly.
• 7:09
  Speaker 1: Now, I want to push on this a bit because
• 7:10
  Speaker 1: the hiring example is largely about historical bias, like Mistakes
• 7:14
  Speaker 1: from decades past haunting us today. But the text points
• 7:18
  Speaker 1: out that bias doesn't just live in dusty corporate archives.
• 7:21
  Speaker 1: It creeps into these systems at the exact moment we
• 7:23
  Speaker 1: start building them today.
• 7:25
  Speaker 2: It's a very active process.
• 7:26
  Speaker 1: Yeah, it's happening right now.
• 7:27
  Speaker 2: It is an actively ongoing problem. The text walks us
• 7:31
  Speaker 2: through the anatomy of an automated system, and it shows
• 7:34
  Speaker 2: how human subjectivity enters at almost every single stage of development.
• 7:39
  Speaker 1: Give me some examples of those stages.
• 7:41
  Speaker 2: Well, it happens during data collection.
• 7:43
  Speaker 1: Okay.
• 7:44
  Speaker 2: It happens when humans create labels for that data. It
• 7:48
  Speaker 2: happens when developers select which features the AI should actually
• 7:51
  Speaker 2: pay attention to.
• 7:52
  Speaker 1: So at every step, basically.
• 7:54
  Speaker 2: And it happens when they define the mathematical thresholds for
• 7:57
  Speaker 2: a final decision.
• 7:58
  Speaker 1: Let's focus heavily on the data collection part, specifically what
• 8:02
  Speaker 1: the text calls sampling bias.
• 8:04
  Speaker 2: Oh, this is a huge one.
• 8:06
  Speaker 1: They use a medical AI example that I found really
• 8:08
  Speaker 1: striking and, frankly, a bit scary.
• 8:10
  Speaker 2: What's fascinating here is how easily this happens, even with
• 8:14
  Speaker 2: the absolute best intentions from world-class researchers.
• 8:17
  Speaker 1: Walk us through it.
• 8:18
  Speaker 2: So suppose you are building a medical AI system to
• 8:21
  Speaker 2: diagnose a specific aggressive disease. Okay.
• 8:24
  Speaker 1: A noble goal.
• 8:25
  Speaker 2: Very noble. You need data to train it, obviously. So
• 8:28
  Speaker 2: you gather patient data from a localized limited population.
• 8:32
  Speaker 1: Like a specific hospital.
• 8:33
  Speaker 2: Exactly. Exactly. maybe three prestigious research hospitals in one major
• 8:39
  Speaker 2: affluent city.
• 8:40
  Speaker 1: Because that's where the researchers work. It's convenient.
• 8:43
  Speaker 2: Right. So the model churns through that data, it becomes
• 8:46
  Speaker 2: incredibly sophisticated, and it performs extremely well.
• 8:50
  Speaker 1: For those specific patients.
• 8:51
  Speaker 2: Exactly. It works great for people who are biologically, demographically,
• 8:56
  Speaker 2: or socioeconomically similar to the patients in that original training data.
• 9:01
  Speaker 1: But then you commercialize it. You package it up and
• 9:04
  Speaker 1: you deploy it somewhere else.
• 9:05
  Speaker 2: And that is where the architecture completely crumbles. It performs
• 9:09
  Speaker 2: significantly worse for populations that are underrepresented in the original data.
• 9:13
  Speaker 1: Because it's never seen them before.
• 9:15
  Speaker 2: Right. The algorithm isn't intentionally malicious. It simply had a
• 9:19
  Speaker 2: massive statistical blind spot.
• 9:21
  Speaker 1: A blind spot that could cost lives.
• 9:23
  Speaker 2: Exactly. Because it was fed less information about certain groups,
• 9:27
  Speaker 2: its predictive power for those groups is fundamentally less reliable.
• 9:30
  Speaker 1: And this is where I want to pause and address you,
• 9:33
  Speaker 1: the listener, directly.
• 9:34
  Speaker 2: It's an important point.
• 9:35
  Speaker 1: Before you ever trust a system that boasts in a
• 9:38
  Speaker 1: press release about being highly accurate, you have to ask
• 9:42
  Speaker 1: one crucial foundational question.
• 9:45
  Speaker 2: Which is?
• 9:45
  Speaker 1: Accurate for whom?
• 9:46
  Speaker 2: Yes, accurate for whom?
• 9:47
  Speaker 1: Because in medicine, biological characteristics differ. Disease presentation differs.
• 9:52
  Speaker 2: Greatly.
• 9:53
  Speaker 1: Healthcare access, nutrition, and environmental factors differ wildly across populations.
• 9:58
  Speaker 2: They absolutely do.
• 9:59
  Speaker 1: A diagnostic model... built on data from a wealthy hospital
• 10:02
  Speaker 1: in Boston might fail spectacularly if it's deployed in a
• 10:06
  Speaker 1: rural clinic in another country.
• 10:08
  Speaker 2: Or even just in an underserved neighborhood a few miles away.
• 10:10
  Speaker 1: Exactly. Just down the street. Which brings us to the
• 10:14
  Speaker 1: dangerous trap of average accuracy.
• 10:16
  Speaker 2: We are deeply conditioned by society to look at aggregate statistics.
• 10:21
  Speaker 2: We just accept them as the whole truth.
• 10:23
  Speaker 1: Like a tech CEO might stand up on a stage.
• 10:26
  Speaker 2: Wearing a black turtleneck.
• 10:28
  Speaker 1: Exactly, wearing a turtleneck. And they declare, our new diagnostic
• 10:32
  Speaker 1: model achieves 95% accuracy across the entire data set.
• 10:36
  Speaker 2: And everyone claps. 95% sounds amazing.
• 10:38
  Speaker 1: Right. In school, that is an A. Who wouldn't want
• 10:42
  Speaker 1: a doctor who is right 95% of the time?
• 10:45
  Speaker 2: It sounds phenomenal on a slide deck, but that overall
• 10:49
  Speaker 2: figure can completely conceal a devastating reality.
• 10:52
  Speaker 1: How so? Because 95 is a big number.
• 10:55
  Speaker 2: We have to look at the statistical distribution. Suppose that
• 10:57
  Speaker 2: 95% overall accuracy breaks down like this.
• 11:01
  Speaker 1: Okay, let's hear the math.
• 11:03
  Speaker 2: It is 98% accurate for a majority demographic group that
• 11:07
  Speaker 2: happens to make up 90% of the data set.
• 11:09
  Speaker 1: Okay, so for the vast majority of people, it's basically perfect. Yes.
• 11:12
  Speaker 2: Yeah. But it is only 60% accurate for a smaller,
• 11:16
  Speaker 2: underrepresented group that makes up the remaining 10%.
• 11:19
  Speaker 1: Wow. Wait, 60% in a medical diagnosis is essentially a
• 11:22
  Speaker 1: coin flip.
• 11:23
  Speaker 2: It is.
• 11:24
  Speaker 1: That is terrifying.
• 11:25
  Speaker 2: For that smaller group, the system is fundamentally broken. It
• 11:28
  Speaker 2: is an absolute hazard.
• 11:30
  Speaker 1: But the CEO gets to say 95%.
• 11:32
  Speaker 2: Exactly. The failure is completely masked by the sheer volume
• 11:36
  Speaker 2: of the majority group's data. That massive group pulls the
• 11:39
  Speaker 2: mathematical average up.
• 11:40
  Speaker 1: It dilutes the failure.
• 11:42
  Speaker 2: This is why responsible AI evaluation requires something called disaggregated testing.
• 11:47
  Speaker 1: Disaggregated testing, meaning break the numbers apart.
• 11:50
  Speaker 2: Exactly. Breaking performance down across relevant subgroups. You cannot let
• 11:56
  Speaker 2: life-altering disparities disappear inside a comforting average.
• 11:59
  Speaker 1: Okay, so a logical listener might hear this and think,
• 12:02
  Speaker 1: all right, I get it. The data is messy. History
• 12:05
  Speaker 1: is biased.
• 12:06
  Speaker 2: Sure.
• 12:06
  Speaker 1: But surely, if we just sit down with the smartest
• 12:09
  Speaker 1: engineers and mathematicians... Put them.
• 12:11
  Speaker 2: In a room.
• 12:12
  Speaker 1: Right, put them in a room. We can fix this
• 12:14
  Speaker 1: by writing the perfect mathematical formula for fairness. We just
• 12:18
  Speaker 1: define what fair means in the code.
• 12:20
  Speaker 2: Right, it seems like the logical next step.
• 12:22
  Speaker 1: But the text reveals something that completely broke my brain.
• 12:26
  Speaker 2: Which is?
• 12:26
  Speaker 1: Fairness defies mathematics.
• 12:29
  Speaker 2: It does. It is not just difficult. It is a
• 12:31
  Speaker 2: proven mathematical impossibility... to achieve universal fairness across all definitions simultaneously.
• 12:38
  Speaker 1: Mathematically impossible. That's a strong statement.
• 12:41
  Speaker 2: And the text uses a brilliant practical example to prove this.
• 12:44
  Speaker 2: Let's look at a bank evaluating loan applications using an
• 12:47
  Speaker 2: automated system.
• 12:48
  Speaker 1: Oh, walk us through this. I really want to take
• 12:50
  Speaker 1: our time here because the way these different definitions of
• 12:54
  Speaker 1: fairness collide is just mind-bending.
• 12:56
  Speaker 2: Okay, so imagine you are the bank's chief ethicist. And
• 13:01
  Speaker 2: you want your new algorithmic loan system to be perfectly fair. OK.
• 13:06
  Speaker 1: I have my Ephesus badge on. I want to be fair.
• 13:08
  Speaker 2: First, you might try a principle called individual fairness.
• 13:12
  Speaker 1: What does that mean?
• 13:13
  Speaker 2: This principle says applicants with similar financial characteristics should receive
• 13:18
  Speaker 2: similar decisions.
• 13:19
  Speaker 1: That makes total sense. Right. If two people have the
• 13:22
  Speaker 1: exact same credit score.
• 13:23
  Speaker 2: The same debt, the same income. Right.
• 13:25
  Speaker 1: They should both get the loan or both be denied,
• 13:27
  Speaker 1: regardless of their background.
• 13:29
  Speaker 2: That seems undeniably fair. That's treating people as individuals.
• 13:33
  Speaker 1: I agree. Problem solved. Let's go to lunch.
• 13:35
  Speaker 2: Not so fast. Because then a sociologist in the room says, wait,
• 13:39
  Speaker 2: historically certain demographic groups were systematically denied wealth-building opportunities like homeownership.
• 13:45
  Speaker 1: True. Redlining and all of that.
• 13:47
  Speaker 2: Exactly. So to be truly fair and correct those historical wrongs,
• 13:53
  Speaker 2: our overall loan approval rates should be identical across demographic groups.
• 13:57
  Speaker 1: Okay, wait, explain that.
• 13:59
  Speaker 2: If we approve 20% of group A, we must approve
• 14:02
  Speaker 2: 20% of group B. This is often called demographic parity.
• 14:06
  Speaker 1: Okay, I see. That is also a highly valid, noble
• 14:09
  Speaker 1: definition of fairness.
• 14:10
  Speaker 2: Right.
• 14:11
  Speaker 1: You're trying to level a playing field that was tilted
• 14:13
  Speaker 1: for centuries.
• 14:14
  Speaker 2: But then a statistician chimes in with a third definition.
• 14:17
  Speaker 1: Oh boy, third one.
• 14:18
  Speaker 2: The statistician says, actually, fairness means ensuring that qualified applicants
• 14:23
  Speaker 2: from any group have equal probabilities of being approved.
• 14:27
  Speaker 1: Meaning, if you're actually a good candidate, your group identity
• 14:30
  Speaker 1: shouldn't lower your chances.
• 14:31
  Speaker 2: Precisely. And finally, the bank's risk manager argues a fourth point.
• 14:35
  Speaker 1: Fourth definition.
• 14:36
  Speaker 2: Yes. The risk manager says, no, fairness means that of
• 14:40
  Speaker 2: the people we do approve, they should all have equal
• 14:43
  Speaker 2: probabilities of actually repaying the loan.
• 14:45
  Speaker 1: Wait, what does that mean in practice?
• 14:47
  Speaker 2: It means a loan given to group A should have
• 14:50
  Speaker 2: the same default rate as a loan given to group B.
• 14:52
  Speaker 1: Wait, hold on. I'm putting on my bank manager hat
• 14:55
  Speaker 1: for a second.
• 14:56
  Speaker 2: Hold on.
• 14:57
  Speaker 1: If my entire job is risk management, why should I
• 15:00
  Speaker 1: care about identical approval rates? I just want to lend
• 15:03
  Speaker 1: money to the people who will actually pay me back.
• 15:05
  Speaker 2: That's the risk manager's exact argument.
• 15:07
  Speaker 1: Isn't forcing identical approval rates are just bad business if
• 15:10
  Speaker 1: the underlying financial realities of those groups are currently different?
• 15:14
  Speaker 2: And that is exactly the tension. The text highlights a harsh,
• 15:18
  Speaker 2: unyielding mathematical reality.
• 15:21
  Speaker 1: Well, let's hear it.
• 15:22
  Speaker 2: When the base rates differ between groups in the real world,
• 15:25
  Speaker 2: these different mathematical definitions of fairness become completely incompatible.
• 15:30
  Speaker 1: Base rates, meaning like existing wealth, historical debt, income distribution.
• 15:36
  Speaker 2: Yes. When those underlying statistical realities are different across groups,
• 15:40
  Speaker 2: the math just clashes.
• 15:42
  Speaker 1: Meaning you literally cannot write code that satisfies all of them.
• 15:45
  Speaker 2: You mathematically cannot. It is impossible. Wow. Let me give
• 15:48
  Speaker 2: you an example. If the average wealth of group A
• 15:51
  Speaker 2: is historically lower than group B.
• 15:53
  Speaker 1: Because of that history we talked about. Right.
• 15:55
  Speaker 2: And you force the algorithm to approve both groups at
• 15:58
  Speaker 2: the exact same 20% rate to achieve demographic parity.
• 16:02
  Speaker 1: Okay.
• 16:03
  Speaker 2: You will inevitably have to approve less financially qualified people
• 16:07
  Speaker 2: from group A and reject more financially qualified people from
• 16:10
  Speaker 2: group B.
• 16:11
  Speaker 1: To make the percentages match.
• 16:12
  Speaker 2: Exactly.
• 16:13
  Speaker 1: But wait, that instantly violates the very first rule of
• 16:16
  Speaker 1: fairness we established.
• 16:17
  Speaker 2: Individual fairness.
• 16:18
  Speaker 1: Right. That people with similar financial characteristics should get similar decisions.
• 16:23
  Speaker 1: If you force the 20%, you are treating two people
• 16:25
  Speaker 1: with the same credit score differently based on their group.
• 16:28
  Speaker 2: Exactly. You fix one metric of fairness and you mathematically
• 16:31
  Speaker 2: break another.
• 16:32
  Speaker 1: It's like a game of whack-a-mole.
• 16:33
  Speaker 2: It really is. You cannot solve human fairness by selecting
• 16:37
  Speaker 2: the right algorithmic equation.
• 16:39
  Speaker 1: So what do we do?
• 16:40
  Speaker 2: A human being, like a bank manager... a regulator, an
• 16:43
  Speaker 2: elected official, an ethicist. They have to actually sit down
• 16:47
  Speaker 2: and make a highly subjective, highly fraught ethical and legal decision.
• 16:52
  Speaker 1: About which definition of fairness to pick.
• 16:55
  Speaker 2: Exactly. Which definition is appropriate for this specific real world context.
• 16:59
  Speaker 1: And this inevitably leads to agonizing choices about error distribution.
• 17:04
  Speaker 2: Yes, because no system is perfect. Every system makes mistakes.
• 17:08
  Speaker 1: But the text brings up this fascinating conceptual contrast between
• 17:12
  Speaker 1: false positives and false negatives.
• 17:14
  Speaker 2: It's a crucial distinction.
• 17:15
  Speaker 1: These aren't just abstract statistics. They are things that happen
• 17:18
  Speaker 1: to actual human beings.
• 17:20
  Speaker 2: Let's define them. A false positive is when the system
• 17:23
  Speaker 2: predicts an event will happen, or a condition exists, and
• 17:27
  Speaker 2: it is wrong.
• 17:28
  Speaker 1: Okay, so it says yes, but the answer is no.
• 17:30
  Speaker 2: Right. And a false negative is when the system predicts
• 17:33
  Speaker 2: something will not happen, or a condition does not exist.
• 17:36
  Speaker 2: And it is wrong.
• 17:37
  Speaker 1: So it says no, but the answer is yes.
• 17:39
  Speaker 2: Exactly. Right.
• 17:40
  Speaker 1: So if we go back to the medical screening system,
• 17:43
  Speaker 1: a false positive means the AI analyzes your scan and
• 17:46
  Speaker 1: tells you that you might have a severe disease.
• 17:48
  Speaker 2: But you're actually perfectly healthy.
• 17:50
  Speaker 1: Right. Which causes tremendous unnecessary worry.
• 17:53
  Speaker 2: Psychological stress, panic.
• 17:55
  Speaker 1: And potentially some really invasive, costly follow-up testing.
• 17:59
  Speaker 2: Yes. It is definitely bad.
• 18:01
  Speaker 1: But a false negative means you have an aggressive tumor.
• 18:05
  Speaker 1: And the AI looks at the scan and tells you
• 18:08
  Speaker 1: that you're perfectly healthy.
• 18:09
  Speaker 2: It fails to identify a disease that is actively harming you.
• 18:12
  Speaker 1: The real-world consequences of those two mathematical errors are wildly different.
• 18:16
  Speaker 2: Completely different. Now compare that medical context to a fraud
• 18:20
  Speaker 2: detection system for your credit card.
• 18:22
  Speaker 1: Okay, let's switch gears.
• 18:23
  Speaker 2: A false positive in fraud detection might mean the algorithm
• 18:27
  Speaker 2: blocks you from buying a coffee on vacation.
• 18:29
  Speaker 1: Because it thinks the out-of-state transaction is suspicious.
• 18:32
  Speaker 2: Right. It's annoying you have to call the bank.
• 18:34
  Speaker 1: But a false negative in that scenario.
• 18:35
  Speaker 2: A false negative means the algorithm fails to spot a
• 18:39
  Speaker 2: hacker and it lets them drain your entire bank account. Oh, man.
• 18:43
  Speaker 1: So in medicine, you'd rather have false positives. Let's do
• 18:46
  Speaker 1: the extra test just to be safe.
• 18:47
  Speaker 2: Yes.
• 18:48
  Speaker 1: In fraud detection, you also want to lean toward false
• 18:51
  Speaker 1: positives to protect the money. Block the coffee just in
• 18:55
  Speaker 1: case it's a hacker.
• 18:56
  Speaker 2: Exactly.
• 18:56
  Speaker 1: But when you escalate this to the high stakes of
• 19:00
  Speaker 1: criminal justice or employment... Or that bank loan we were
• 19:03
  Speaker 1: just talking about.
• 19:04
  Speaker 2: It gets incredibly messy.
• 19:06
  Speaker 1: Developers have to actively decide which type of error is
• 19:09
  Speaker 1: the most costly. And importantly, who bears the cost of
• 19:12
  Speaker 1: those errors?
• 19:13
  Speaker 2: That is a critical insight. You could have an AI
• 19:16
  Speaker 2: system that boasts identical overall accuracy for two different demographic groups.
• 19:22
  Speaker 1: But it produces entirely different types of mistakes for each group.
• 19:26
  Speaker 2: Exactly. One group might get hit with all the false positives.
• 19:30
  Speaker 1: Like getting unfairly flagged as high risk for a loan
• 19:33
  Speaker 1: when they actually would pay it back.
• 19:34
  Speaker 2: Right. While the other group gets all the false negatives.
• 19:37
  Speaker 2: Getting approved for a loan when they actually can't afford
• 19:39
  Speaker 2: the payments.
• 19:40
  Speaker 1: Which sets them up for bankruptcy.
• 19:42
  Speaker 2: The overall accuracy is exactly the same, but the lived
• 19:45
  Speaker 2: experience of the automated system is entirely different. Wow. This
• 19:49
  Speaker 2: is why just looking at a single accuracy metric is
• 19:52
  Speaker 2: a terrible measure of fairness.
• 19:55
  Speaker 1: It hides all the actual harm. Okay, so since we
• 19:58
  Speaker 1: can't mathematically define fairness, and since the historical data is
• 20:02
  Speaker 1: inherently biased, a very logical listener might be sitting there thinking.
• 20:06
  Speaker 2: Thinking they have the silver bullet. Right. Right.
• 20:08
  Speaker 1: They might think, OK, I have the solution. Let's just
• 20:10
  Speaker 1: blind the algorithm.
• 20:12
  Speaker 2: Yes.
• 20:12
  Speaker 1: Just delete race, delete gender, delete religion, delete age. Remove
• 20:17
  Speaker 1: any sensitive demographic data from the database entirely before we
• 20:20
  Speaker 1: train the AI.
• 20:21
  Speaker 2: Sounds foolproof.
• 20:22
  Speaker 1: If the AI doesn't know your race, it can't possibly
• 20:25
  Speaker 1: be racist.
• 20:26
  Speaker 2: Right.
• 20:27
  Speaker 1: But that brings us to what the text calls the
• 20:29
  Speaker 1: absolute illusion of the blind algorithm.
• 20:32
  Speaker 2: It is perhaps the single most persistent myth in algorithmic design.
• 20:36
  Speaker 1: People really cling to it.
• 20:37
  Speaker 2: They do. Organizations frequently scrub obvious sensitive attributes from their
• 20:42
  Speaker 2: data sets and just pat themselves on the back.
• 20:44
  Speaker 1: Like problem solved.
• 20:46
  Speaker 2: Exactly. Assuming the resulting model is purely objective, but they're
• 20:51
  Speaker 2: entirely ignoring the concept of proxy variables.
• 20:54
  Speaker 1: Because the AI always finds a way?
• 20:56
  Speaker 2: It finds a way because human lives are deeply, intricately interconnected.
• 21:00
  Speaker 1: What's a proxy variable? Exactly.
• 21:02
  Speaker 2: Other seemingly boring variables act as statistical stand-ins or proxies
• 21:07
  Speaker 2: for the sensitive data you just deleted.
• 21:09
  Speaker 1: Okay, give me an example.
• 21:10
  Speaker 2: Postal codes or zip codes are the classic textbook example.
• 21:13
  Speaker 1: Oh, right. Because of redlining and segregation. Yes.
• 21:16
  Speaker 2: Because of how our cities were historically segregated through policies
• 21:21
  Speaker 2: like redlining and how they subsequently developed over decades, your
• 21:25
  Speaker 2: postal code is incredibly strongly correlated with your racial and
• 21:29
  Speaker 2: socioeconomic demographics.
• 21:31
  Speaker 1: So if the AI denies a loan based on a
• 21:34
  Speaker 1: zip code.
• 21:35
  Speaker 2: It's effectively denying it based on demographics.
• 21:38
  Speaker 1: Even if the column labeled race was deleted from the spreadsheet.
• 21:41
  Speaker 2: Precisely. Yeah. And the danger of modern machine learning is
• 21:44
  Speaker 2: that it doesn't stop at obvious things like zip codes.
• 21:47
  Speaker 1: It goes deeper.
• 21:48
  Speaker 2: Way deeper. Employment history can reflect structural differences in opportunity.
• 21:53
  Speaker 2: Purchasing behavior, the level of education, browser history.
• 21:56
  Speaker 1: It's just scooping up everything.
• 21:58
  Speaker 2: Even the specific vocabulary, slang, and syntax you use in
• 22:01
  Speaker 2: online forums.
• 22:02
  Speaker 1: Just the words you type.
• 22:03
  Speaker 2: Thousands of seemingly neutral mundane variables can indirectly reveal your
• 22:08
  Speaker 2: sensitive characteristics.
• 22:09
  Speaker 1: Modern machine learning is a pattern recognition monster. It operates
• 22:13
  Speaker 1: in this high dimensional space that our human brains can't
• 22:15
  Speaker 1: even visualize.
• 22:16
  Speaker 2: It's beyond human comprehension.
• 22:18
  Speaker 1: You pull out the big obvious thread like race or
• 22:22
  Speaker 1: gender and the AI just weaves the pattern right back
• 22:25
  Speaker 1: together using a thousand tiny invisible threads.
• 22:29
  Speaker 2: Exactly. Removing the variable does absolutely nothing to remove its influence.
• 22:34
  Speaker 1: It just hides it.
• 22:35
  Speaker 2: This introduces a related concept that I find deeply compelling,
• 22:39
  Speaker 2: which the text details beautifully.
• 22:41
  Speaker 1: What's that?
• 22:42
  Speaker 2: The measurement trap.
• 22:42
  Speaker 1: The measurement trap. Okay, explain that.
• 22:44
  Speaker 2: Because an AI system always needs a target variable. It
• 22:48
  Speaker 2: needs a concrete number to optimize for. We frequently use flawed,
• 22:53
  Speaker 2: oversimplified proxies for complex, abstract human concepts.
• 22:58
  Speaker 1: Because you can't measure a concept. Right.
• 23:00
  Speaker 2: Let's break down the health care example from the text
• 23:02
  Speaker 2: because this really cemented the concept for me.
• 23:04
  Speaker 1: Let's do it. And again, this isn't hypothetical. This is
• 23:06
  Speaker 1: based on real world systems.
• 23:08
  Speaker 2: Imagine a massive health care provider wants to predict which
• 23:12
  Speaker 2: of its millions of patients will need the most additional
• 23:15
  Speaker 2: medical support next year.
• 23:17
  Speaker 1: So they can proactively allocate resources, get ahead of the curve.
• 23:20
  Speaker 2: Exactly. The abstract concept they want to measure is health need.
• 23:23
  Speaker 1: But you can't easily quantify need.
• 23:26
  Speaker 2: No, it's qualitative. It's a feeling. It's a physical state.
• 23:30
  Speaker 2: The developers look for a proxy they can easily measure
• 23:33
  Speaker 2: in a database. A number. Right. And they settle on
• 23:36
  Speaker 2: historical health care spending. Oh. Their foundational assumption is patients
• 23:42
  Speaker 2: who cost the health care system more money must be
• 23:45
  Speaker 2: the ones who need the most care.
• 23:47
  Speaker 1: I mean, it sounds totally logical if you're just looking
• 23:49
  Speaker 1: at a spreadsheet.
• 23:50
  Speaker 2: It really does.
• 23:51
  Speaker 1: Money spent equals care given, which equals sickness.
• 23:55
  Speaker 2: Right. But the reality of the health care system shatters
• 23:58
  Speaker 2: that assumption. How so? You can have two patients with
• 24:00
  Speaker 2: the exact same severe chronic illness.
• 24:03
  Speaker 1: Okay, patient A and patient B.
• 24:05
  Speaker 2: Patient A has comprehensive insurance. They live in the suburbs
• 24:09
  Speaker 2: near a major research hospital, have a high income, and
• 24:12
  Speaker 2: can take time off work.
• 24:13
  Speaker 1: So they have great access.
• 24:14
  Speaker 2: They go to the doctor frequently. They get preventative scans.
• 24:17
  Speaker 2: So their historical spending is very high.
• 24:18
  Speaker 1: Makes sense. What about patient B?
• 24:21
  Speaker 2: Patient B has the exact same illness, but they lack insurance.
• 24:26
  Speaker 2: live in a rural medical desert, or work three jobs,
• 24:29
  Speaker 2: and just can't afford to take a Tuesday off to
• 24:30
  Speaker 2: see a specialist.
• 24:31
  Speaker 1: Wow. Okay, so their historical spending is very low.
• 24:35
  Speaker 2: Exactly.
• 24:36
  Speaker 1: Oh, wow. So when the AI looks at the data,
• 24:39
  Speaker 1: it says patient A costs a lot of money. They
• 24:42
  Speaker 1: must need intervention.
• 24:44
  Speaker 2: And patient B costs almost nothing. They must be perfectly healthy.
• 24:47
  Speaker 1: Even though they have the exact same illness.
• 24:49
  Speaker 2: Yes. The AI isn't learning patterns of illness at all.
• 24:53
  Speaker 2: It is learning patterns of health care access and socioeconomic privilege.
• 24:57
  Speaker 1: That is incredible and terrifying.
• 25:00
  Speaker 2: It flawlessly predicts the variable it was given, which is spending,
• 25:03
  Speaker 2: not the concept the humans actually intended, which is need.
• 25:07
  Speaker 1: Here's where it gets really interesting, because this isn't just
• 25:09
  Speaker 1: a health care problem. It happens everywhere.
• 25:12
  Speaker 2: Everywhere we use data.
• 25:13
  Speaker 1: The text brings up an example in education that every student, parent,
• 25:17
  Speaker 1: or former student can instantly relate to.
• 25:19
  Speaker 2: The grading algorithm. Yes.
• 25:21
  Speaker 1: Suppose a school district uses an algorithm to predict a
• 25:24
  Speaker 1: student's potential or academic ability to decide who gets into
• 25:29
  Speaker 1: an advanced gifted program.
• 25:30
  Speaker 2: A very common scenario. Right.
• 25:32
  Speaker 1: They use previous test scores and grades as the measurement,
• 25:35
  Speaker 1: but treating grades as a perfect, objective measurement of pure
• 25:39
  Speaker 1: cognitive ability completely ignores the reality of being a human student.
• 25:44
  Speaker 2: Grades are a proxy. They reflect knowledge, certainly. But what
• 25:48
  Speaker 2: else do they reflect?
• 25:49
  Speaker 1: Lots of things.
• 25:50
  Speaker 2: They reflect whether a student has good attendance, which might
• 25:53
  Speaker 2: depend on whether their parents have a reliable car.
• 25:55
  Speaker 1: Exactly.
• 25:56
  Speaker 2: They reflect teacher expectations and implicit biases in grading subjective essays.
• 26:01
  Speaker 1: They reflect whether the family can afford private tutoring or
• 26:05
  Speaker 1: whether the student is learning in a second language.
• 26:07
  Speaker 2: Or even whether they had a quiet room to sleep
• 26:09
  Speaker 2: in the night before the exam.
• 26:11
  Speaker 1: The AI is incredibly aggressively literal.
• 26:14
  Speaker 2: It doesn't know what the abstract concept of potential is.
• 26:16
  Speaker 1: It only knows what a mathematical 92% is. Right.
• 26:19
  Speaker 2: If we optimize the wrong measurement, the model becomes terrifyingly
• 26:24
  Speaker 2: effective at achieving the completely wrong objective.
• 26:28
  Speaker 1: And up to this point in our discussion, we've mostly
• 26:30
  Speaker 1: been looking at algorithms making static predictions about the world.
• 26:34
  Speaker 2: Just reading the data.
• 26:35
  Speaker 1: Right. They look at historical data, they make a guess,
• 26:38
  Speaker 1: and they move on. But we have to ask the
• 26:40
  Speaker 1: bigger question.
• 26:41
  Speaker 2: Which is?
• 26:41
  Speaker 1: What happens when the algorithm's predictions start actively changing the
• 26:46
  Speaker 1: world they are supposedly just observing?
• 26:49
  Speaker 2: This is where we cross the line from passive observation
• 26:52
  Speaker 2: into active reality distortion.
• 26:54
  Speaker 1: That sounds like a sci-fi movie, reality distortion.
• 26:57
  Speaker 2: It really does, but it's happening. The text dives deeply
• 27:00
  Speaker 2: into the concept of feedback loops.
• 27:02
  Speaker 1: Feedback loops.
• 27:03
  Speaker 2: And it uses predictive policing as a core conceptual example,
• 27:07
  Speaker 2: which is just vital to understand.
• 27:09
  Speaker 1: Okay, let's unpack predictive policing.
• 27:11
  Speaker 2: If we connect this to the bigger picture, a feedback
• 27:14
  Speaker 2: loop happens when an algorithm's decisions generate the very data
• 27:18
  Speaker 2: that will be used to train it in the future.
• 27:20
  Speaker 1: It is an endless self-reinforcing cycle.
• 27:23
  Speaker 2: Exactly. In the predictive policing example, imagine an algorithm analyzes
• 27:28
  Speaker 2: historical arrest data and it flags a specific neighborhood as
• 27:33
  Speaker 2: a high-risk area for future crime.
• 27:35
  Speaker 1: So the police command looks at the digital map, sees
• 27:38
  Speaker 1: a glowing red hotspot over that neighborhood. And they logically
• 27:42
  Speaker 1: send more police cruisers to patrol that specific area.
• 27:46
  Speaker 2: Naturally. That is how they use the tool. It's doing
• 27:48
  Speaker 2: what it's supposed to do.
• 27:49
  Speaker 1: But because there is now a highly increased concentrated police
• 27:53
  Speaker 1: presence in that area.
• 27:54
  Speaker 2: Officers are going to observe and record far more minor incidents.
• 27:58
  Speaker 2: Traffic infractions, loitering, petty possession.
• 28:01
  Speaker 1: Things that might have gone completely unnoticed if they weren't
• 28:03
  Speaker 1: patrolling that specific street so heavily.
• 28:06
  Speaker 2: Exactly. And all those new, newly recorded incidents, those new arrests,
• 28:10
  Speaker 2: go right back into the central database.
• 28:12
  Speaker 1: Oh, man. So the new data feeds back into the algorithm.
• 28:15
  Speaker 2: Yes. The algorithm processes the newly inflated numbers and says, look,
• 28:19
  Speaker 2: my prediction was incredibly accurate. Crime is indeed up in
• 28:23
  Speaker 2: this area. We need to send even more patrols.
• 28:25
  Speaker 1: It proved itself right.
• 28:27
  Speaker 2: The algorithm is no longer simply observing reality. Its recommendations
• 28:31
  Speaker 2: are actively manufacturing reality.
• 28:34
  Speaker 1: It creates a localized surveillance state. Based on its own
• 28:37
  Speaker 1: initial guess.
• 28:38
  Speaker 2: It's a self-fulfilling prophecy coded in Python. Your initial bias
• 28:43
  Speaker 2: creates the data that justifies the bias.
• 28:46
  Speaker 1: And you can extrapolate this exact same feedback loop concept
• 28:50
  Speaker 1: to almost everything we interact with digitally today.
• 28:53
  Speaker 2: Absolutely everything.
• 28:54
  Speaker 1: Think about recommendation systems on social media or YouTube or
• 28:57
  Speaker 1: streaming platforms. The system recommends a video to you. Because
• 29:01
  Speaker 1: the algorithm pushed it to the top of everyone's feed,
• 29:03
  Speaker 1: it gets a massive amount of clicks.
• 29:05
  Speaker 2: Right.
• 29:06
  Speaker 1: The system sees all those clicks, assumes the content is
• 29:09
  Speaker 1: highly organically popular, and recommends it to even more people.
• 29:12
  Speaker 2: It single-handedly manufactures its own popularity. It wasn't popular until
• 29:16
  Speaker 2: the machine said it was.
• 29:17
  Speaker 1: And the same dynamic occurs in hiring algorithms, circling all
• 29:21
  Speaker 1: the way back to our very first example.
• 29:23
  Speaker 2: Yes, the resume screener.
• 29:24
  Speaker 1: If a model consistently favors and hires candidates with a
• 29:27
  Speaker 1: specific background, those exact candidates become the employees whose future
• 29:31
  Speaker 1: performance data is collected.
• 29:33
  Speaker 2: The model's own biased choices... literally shape the composition of
• 29:37
  Speaker 2: its next training data set.
• 29:39
  Speaker 1: It locks out anyone different, so it never gets data
• 29:41
  Speaker 1: showing that someone different could have succeeded.
• 29:44
  Speaker 2: I really want to frame this directly for the listener.
• 29:46
  Speaker 1: Please do.
• 29:47
  Speaker 2: Your past behavior didn't just predict your future. The algorithm's
• 29:51
  Speaker 2: prediction literally forced your future.
• 29:54
  Speaker 1: It built the walls around you based on a statistical guess.
• 29:57
  Speaker 2: Which is precisely why we cannot evaluate automated systems as
• 30:01
  Speaker 2: if they are isolated little calculators sitting on a desk
• 30:04
  Speaker 2: in a sterile lab.
• 30:05
  Speaker 1: They aren't just doing math in a vacuum.
• 30:08
  Speaker 2: We have to examine how they dynamically interact with real
• 30:11
  Speaker 2: human behavior, institutions, and society over time. They are social
• 30:16
  Speaker 2: systems just as much as they are technical systems.
• 30:18
  Speaker 1: And this idea of AI actively shaping culture and reality...
• 30:22
  Speaker 1: leads us directly into the massive world-eating systems that absolutely
• 30:26
  Speaker 1: everyone is talking about right now.
• 30:28
  Speaker 2: You mean generative AI.
• 30:29
  Speaker 1: Generative AI and computer vision. We're moving way beyond simple
• 30:32
  Speaker 1: spreadsheets of data here.
• 30:34
  Speaker 2: We really are. Large language models, or LLMs, represent a
• 30:38
  Speaker 2: whole new frontier of this exact problem.
• 30:40
  Speaker 1: Tell me about it.
• 30:41
  Speaker 2: These systems don't just look at a few thousand rows
• 30:44
  Speaker 2: of loan data. They learn from truly enormous collections of text.
• 30:48
  Speaker 1: Like the whole Internet.
• 30:50
  Speaker 2: Essentially scraping the entire public Internet, scanning millions of books, articles,
• 30:54
  Speaker 2: forums and social media posts.
• 30:57
  Speaker 1: And as we all know, those sources are absolutely saturated
• 31:00
  Speaker 1: with human stereotypes, cultural assumptions, historical inequalities. And deeply conflicting
• 31:07
  Speaker 1: toxic perspectives.
• 31:08
  Speaker 2: The internet is not a friendly place.
• 31:10
  Speaker 1: I'm bringing back the parrot analogy, but this parrot hasn't
• 31:12
  Speaker 1: just been listening to a polite corporate HR department.
• 31:15
  Speaker 2: No, this parrot has read the entire internet.
• 31:17
  Speaker 1: Which includes some of the darkest, most hateful places on earth.
• 31:21
  Speaker 2: And that is a massive engineering headache for the people
• 31:24
  Speaker 2: building these models. Developers try to use filtering, reinforcement learning.
• 31:29
  Speaker 2: and fine-tuning to reduce harmful outputs.
• 31:31
  Speaker 1: Trying to teach the parrot some manner.
• 31:33
  Speaker 2: Right. But completely eliminating bias from an LLM is extraordinarily
• 31:38
  Speaker 2: difficult because language itself is an inherently flawed reflection of
• 31:42
  Speaker 2: human culture.
• 31:43
  Speaker 1: The text points out something fascinating about this struggle. Even
• 31:46
  Speaker 1: defining what constitutes undesirable bias in a language model is
• 31:51
  Speaker 1: incredibly contentious.
• 31:52
  Speaker 2: It's a huge debate.
• 31:53
  Speaker 1: Let's say you build an LLM that absolutely refuses to
• 31:57
  Speaker 1: acknowledge any statistical, cultural, or historical differences between groups of people.
• 32:02
  Speaker 1: It treats everyone as perfectly identical.
• 32:05
  Speaker 2: Okay, colorblind, essentially.
• 32:07
  Speaker 1: Right. That model becomes completely inaccurate and utterly useless for
• 32:11
  Speaker 1: understanding the real world, history, or sociology.
• 32:14
  Speaker 2: Because the world isn't identical. But on the flip side
• 32:17
  Speaker 2: of that tightrope, a model that just bluntly repeats every
• 32:20
  Speaker 2: observed statistical difference in its training data without providing any context, nuance,
• 32:25
  Speaker 2: or historical background.
• 32:26
  Speaker 1: What happens then?
• 32:27
  Speaker 2: It ends up validating and reinforcing harmful stereotypes. Responsible generative
• 32:32
  Speaker 2: AI has to walk this incredibly thin, precarious tightrope.
• 32:36
  Speaker 1: Between acknowledging relevant factual differences in the human experience and
• 32:40
  Speaker 1: rejecting unjustified generalization?
• 32:43
  Speaker 2: It's a nearly impossible balance to strike.
• 32:45
  Speaker 1: And this complexity extends beyond text into computer vision. The
• 32:50
  Speaker 1: text mentions how facial recognition performance can vary wildly.
• 32:54
  Speaker 2: Oh, significantly.
• 32:55
  Speaker 1: But I want to dig into why. It's not just demographics.
• 32:58
  Speaker 1: It's a combination of physics and training data.
• 33:01
  Speaker 2: How so?
• 33:01
  Speaker 1: Well, cameras were historically optimized for lighter skin tones in
• 33:05
  Speaker 1: terms of light calibration. Add in bad lighting, cheap camera sensors,
• 33:09
  Speaker 1: or low image resolution, and the system fails.
• 33:12
  Speaker 2: That's a huge technical flaw. Right.
• 33:14
  Speaker 1: Now, if my phone's facial recognition fails to unlock because
• 33:17
  Speaker 1: the lighting in my room is bad, it's annoying. I
• 33:20
  Speaker 1: just type in my passcode.
• 33:21
  Speaker 2: No real harm done.
• 33:22
  Speaker 1: But if that exact same flawed computer vision is used
• 33:25
  Speaker 1: in law enforcement to match a suspect to a grainy
• 33:28
  Speaker 1: security camera... or in border control, or in banking identity verification.
• 33:33
  Speaker 2: Then a mathematical error is no longer just an inconvenience.
• 33:36
  Speaker 1: It's a denial of a fundamental right. It's a false
• 33:39
  Speaker 1: accusation that can ruin a life.
• 33:40
  Speaker 2: Which brings us to a really chilling extrapolation in the
• 33:43
  Speaker 2: text regarding extreme personalization.
• 33:45
  Speaker 1: Extreme personalization. Like targeted ads.
• 33:49
  Speaker 2: More than just ads. AI systems can now analyze thousands
• 33:52
  Speaker 2: upon thousands of variables about you. Your location, your browsing habits,
• 33:56
  Speaker 2: your purchasing history. The time you spend hovering over an image.
• 34:00
  Speaker 1: Just lingering on a picture.
• 34:02
  Speaker 2: Yes. To create a highly personalized, invisible, algorithmic environment uniquely
• 34:08
  Speaker 2: tailored to you.
• 34:09
  Speaker 1: The hyper-curated feed. It sounds great on a shopping app
• 34:12
  Speaker 1: when it recommends the perfect pair of shoes, but the
• 34:15
  Speaker 1: sources point out a really chilling side effect.
• 34:17
  Speaker 2: It is.
• 34:18
  Speaker 1: When everyone in the world is receiving a completely unique,
• 34:21
  Speaker 1: individualized feed... Detecting discrimination becomes almost functionally impossible.
• 34:26
  Speaker 2: Because there is no baseline anymore.
• 34:28
  Speaker 1: Exactly. In the past, if a newspaper ran an ad
• 34:31
  Speaker 1: for a high-paying executive job, everyone who bought the paper
• 34:35
  Speaker 1: that Sunday saw the ad.
• 34:36
  Speaker 2: We all share the same reality.
• 34:37
  Speaker 1: Right. If the company subsequently refused to hire certain types
• 34:41
  Speaker 1: of people, regulators or journalists could eventually spot the pattern.
• 34:44
  Speaker 2: But today?
• 34:45
  Speaker 1: Today, if digital ads for luxury housing, lucrative employment, or
• 34:50
  Speaker 1: prime financial services are distributed unevenly based on hidden traits
• 34:54
  Speaker 1: the AI has inferred about you.
• 34:56
  Speaker 2: You will never even know about the opportunities you were denied.
• 34:59
  Speaker 1: You can't complain to a regulator about not getting a
• 35:02
  Speaker 1: job you never knew existed.
• 35:03
  Speaker 2: That's haunting.
• 35:05
  Speaker 1: Algorithmic fairness isn't just about the decisions made against you.
• 35:08
  Speaker 1: It's about visibility. It's about what the algorithm allows you
• 35:11
  Speaker 1: to see in the first place.
• 35:12
  Speaker 2: That is a profound structural shift in how opportunity is
• 35:17
  Speaker 2: distributed in modern society.
• 35:19
  Speaker 1: It really is.
• 35:19
  Speaker 2: And it begs the ultimate question. If the math for
• 35:23
  Speaker 2: universal fairness is impossible, if the historical data is hopelessly biased,
• 35:28
  Speaker 2: and if extreme personalization hides all the evidence of discrimination
• 35:32
  Speaker 2: from the public eye.
• 35:34
  Speaker 1: How on earth do we hold these systems and the
• 35:36
  Speaker 1: companies that build them accountable?
• 35:38
  Speaker 2: Which brings us right to the door of governance.
• 35:40
  Speaker 1: And what I like to call the illusion of the
• 35:42
  Speaker 1: human in the loop.
• 35:43
  Speaker 2: Oh, this is such a big buzzword right now.
• 35:46
  Speaker 1: Whenever there's a major controversy about an AI system being
• 35:49
  Speaker 1: unfair or making a catastrophic mistake, The tech company's first
• 35:53
  Speaker 1: public defense is almost always, don't worry, we don't let
• 35:56
  Speaker 1: the machine decide on its own.
• 35:58
  Speaker 2: We have a human in the loop.
• 36:00
  Speaker 1: Right. A real person reviews the machine's recommendations.
• 36:04
  Speaker 2: It is consistently offered as this ultimate comforting guarantee of safety.
• 36:08
  Speaker 1: But the text deeply critiques this practice.
• 36:11
  Speaker 2: It does. Because just physically placing a human somewhere in
• 36:15
  Speaker 2: the workflow does not guarantee actual oversight.
• 36:19
  Speaker 1: What do you mean by that?
• 36:21
  Speaker 2: Imagine a human reviewer sitting in a massive moderation center
• 36:24
  Speaker 2: or a warehouse or even a busy clinic. They are
• 36:28
  Speaker 2: tasked with processing hundreds of automated recommendations every single hour.
• 36:34
  Speaker 1: They're just staring at a screen for eight hours, clicking approve, approve, approve,
• 36:37
  Speaker 1: just to keep up with the sheer volume.
• 36:39
  Speaker 2: Exactly. And let's take that further. Psychologists call this automation bias.
• 36:43
  Speaker 1: Automation bias.
• 36:44
  Speaker 2: Humans naturally begin to trust the machine's judgment over their own,
• 36:47
  Speaker 2: especially if the software interface strongly encourages them to just
• 36:51
  Speaker 2: accept the AI's guess with the big green button.
• 36:54
  Speaker 1: It's just easier to click yes.
• 36:55
  Speaker 2: Right. And if that human is penalized by their manager
• 36:58
  Speaker 2: for slowing down their quota to investigate a complicated case,
• 37:02
  Speaker 2: that human oversight is purely ceremonial.
• 37:05
  Speaker 1: They aren't analyzing anything.
• 37:06
  Speaker 2: They are just a biological rubber stamp for the algorithm.
• 37:09
  Speaker 1: It's like when we all scroll to the bottom of
• 37:12
  Speaker 1: a 50-page terms and conditions agreement and just click, I agree,
• 37:15
  Speaker 1: without reading a single word.
• 37:17
  Speaker 2: We are technically the human in the loop. but we
• 37:20
  Speaker 2: aren't providing any actual oversight.
• 37:22
  Speaker 1: To be effective, the text emphasizes that the human reviewer
• 37:26
  Speaker 1: needs time.
• 37:27
  Speaker 2: They need actual authority.
• 37:29
  Speaker 1: And most importantly, they need the psychological and professional safety
• 37:32
  Speaker 1: to override the machine without getting fired for missing their
• 37:36
  Speaker 1: productivity metrics.
• 37:38
  Speaker 2: Otherwise, the human simply becomes another mechanical cog in the
• 37:41
  Speaker 2: automation machinery rather than an independent safeguard.
• 37:45
  Speaker 1: So what does this all mean? If the human in
• 37:47
  Speaker 1: the loop is largely ceremonial, People demand that the AI
• 37:51
  Speaker 1: itself explain its decisions.
• 37:53
  Speaker 2: Explainable AI. Or XAI. Right.
• 37:56
  Speaker 1: This is a big push in the industry. But the
• 37:58
  Speaker 1: sources are very clear about the limits of transparency.
• 38:02
  Speaker 2: Extremely clear.
• 38:02
  Speaker 1: The text argues that giving a purely technical explanation is
• 38:06
  Speaker 1: practically useless.
• 38:07
  Speaker 2: Completely useless to the average person.
• 38:10
  Speaker 1: If a bank denies my loan, and I demand to
• 38:12
  Speaker 1: know why, and they send me an automated report saying,
• 38:16
  Speaker 1: feature 4,827, increase the model's risk score by 12%, I'm
• 38:21
  Speaker 1: going to look at them like they have two heads.
• 38:22
  Speaker 2: Right, that tells me absolutely nothing about my life or
• 38:25
  Speaker 2: what I need to fix.
• 38:26
  Speaker 1: Because a technical explanation is not a meaningful explanation.
• 38:29
  Speaker 2: We have to understand how these models work under the
• 38:31
  Speaker 2: hood to see why.
• 38:33
  Speaker 1: Okay, open the hood. Let's look.
• 38:36
  Speaker 2: Many modern machine learning models, especially deep neural networks, contain
• 38:41
  Speaker 2: millions or even billions of parameters.
• 38:44
  Speaker 1: Billions?
• 38:44
  Speaker 2: Imagine a massive, incomprehensible web of thousands of hidden mathematical
• 38:49
  Speaker 2: dials all turning slightly in response to each other.
• 38:52
  Speaker 1: Just a giant wall of dials.
• 38:54
  Speaker 2: The internal math of those hidden layers does not neatly
• 38:57
  Speaker 2: map onto recognizable human concepts like debt to income ratio
• 39:01
  Speaker 2: or work ethic.
• 39:02
  Speaker 1: So it's not thinking in English.
• 39:03
  Speaker 2: Not at all. Which is why you can't just publish
• 39:05
  Speaker 2: the source code online and call it a day. A
• 39:08
  Speaker 2: tech company might say, we have perfect transparency. Our code
• 39:10
  Speaker 2: is open source.
• 39:12
  Speaker 1: But unless you are a machine learning savant with a
• 39:14
  Speaker 1: supercomputer in your garage, staring at millions of individual numerical
• 39:18
  Speaker 1: weights won't tell you why you didn't get the job.
• 39:21
  Speaker 2: It's transparency without legibility. You can see it, but you
• 39:25
  Speaker 2: can't read it.
• 39:25
  Speaker 1: This is why the text strongly emphasizes the need for
• 39:29
  Speaker 1: meaningful transparency.
• 39:30
  Speaker 2: Things like model cards.
• 39:32
  Speaker 1: Model cards. I love this concept. Think of a model
• 39:35
  Speaker 1: card as a detailed nutritional label for an algorithm.
• 39:39
  Speaker 2: That's a great way to think about it.
• 39:40
  Speaker 1: It explains in plain, non-technical language what was the specific
• 39:45
  Speaker 1: training data used? What are the known statistical limitations?
• 39:49
  Speaker 2: What are the intended uses of this model?
• 39:51
  Speaker 1: And crucially, where should it absolutely not be used?
• 39:54
  Speaker 2: But even with great documentation, the absolute non-negotiable necessity that
• 40:00
  Speaker 2: text points to is appeal mechanisms.
• 40:03
  Speaker 1: Yes, the appeal. Because data can just be wrong.
• 40:06
  Speaker 2: Identities get mixed up on credit reports constantly.
• 40:08
  Speaker 1: Life circumstances change in ways a database just can't capture.
• 40:11
  Speaker 2: And without a robust appeal process where a human can
• 40:15
  Speaker 2: actually intervene, investigate, and override the system. A tiny change
• 40:19
  Speaker 2: Stupid data entry error transforms into a rigid, life-altering decision.
• 40:24
  Speaker 1: And implementing all of these safeguards, like model cards, meaningful explanations,
• 40:29
  Speaker 1: robust appeal processes.
• 40:31
  Speaker 2: It requires balancing multiple competing interests.
• 40:33
  Speaker 1: Which leads us to the grand trade-offs of responsible AI design.
• 40:38
  Speaker 1: We have to talk about metrics.
• 40:40
  Speaker 2: Ah, yes. The metrics.
• 40:42
  Speaker 1: The text delves deeply into the concept of the objective function.
• 40:46
  Speaker 2: This is fundamental.
• 40:47
  Speaker 1: For the listener, an objective function is basically the target
• 40:50
  Speaker 1: you give the AI. An AI system will relentlessly optimize
• 40:53
  Speaker 1: exactly what its human designers ask it to optimize.
• 40:56
  Speaker 2: And the metrics we choose are incredibly powerful, but also
• 41:00
  Speaker 2: incredibly limited.
• 41:01
  Speaker 1: The example of the call center is perfect for understanding
• 41:03
  Speaker 1: this mechanism.
• 41:04
  Speaker 2: Walk us through the call center.
• 41:06
  Speaker 1: Suppose a massive corporation wants to use an AI management
• 41:09
  Speaker 1: system to schedule and monitor its customer service employees.
• 41:12
  Speaker 2: Sounds very corporate.
• 41:13
  Speaker 1: The objective function, the singular metric of success given to
• 41:16
  Speaker 1: the AI, is defined mathematically as minimize the average time
• 41:20
  Speaker 1: spent on a call.
• 41:22
  Speaker 2: Which makes total sense to an executive, because faster calls
• 41:25
  Speaker 2: mean you can handle more volume, which means you need
• 41:27
  Speaker 2: fewer employees.
• 41:28
  Speaker 1: Which saves the company millions of dollars.
• 41:31
  Speaker 2: Right. So the AI goes to work.
• 41:32
  Speaker 1: It monitors everyone, and it becomes extremely ruthless and effective
• 41:36
  Speaker 1: at reducing average call duration.
• 41:38
  Speaker 2: On paper, it seems like a massive success. The charts
• 41:41
  Speaker 2: look great. The time goes down.
• 41:43
  Speaker 1: But what is the hidden cost? Employees are now terrified
• 41:47
  Speaker 1: of taking too long. They start rushing customers off the phone.
• 41:50
  Speaker 2: They provide terrible, incomplete service.
• 41:52
  Speaker 1: And worse, they actively avoid picking up complex cases because
• 41:57
  Speaker 1: resolving a difficult problem would ruin their daily metrics and
• 42:00
  Speaker 1: get them penalized by the algorithm.
• 42:02
  Speaker 2: The model perfectly achieved the target.
• 42:04
  Speaker 1: But the target was incomplete. The AI couldn't quantify customer
• 42:08
  Speaker 1: satisfaction or employee burnouts.
• 42:11
  Speaker 2: So to the optimization system, those human concepts basically didn't exist.
• 42:16
  Speaker 1: It optimized for speed at the absolute cost of quality.
• 42:19
  Speaker 2: This raises an important question. How do we mathematically balance
• 42:23
  Speaker 2: concepts that pull in entirely different directions?
• 42:26
  Speaker 1: The text discusses the intense, ongoing tension between fairness and privacy,
• 42:31
  Speaker 1: for example.
• 42:32
  Speaker 2: This is a massive headache for developers right now.
• 42:34
  Speaker 1: I love this part. You might want to rigorously audit
• 42:37
  Speaker 1: your shiny new AI system for racial or gender bias
• 42:40
  Speaker 1: to ensure it's fair.
• 42:42
  Speaker 2: Right. To see if it's biased, you have to test
• 42:44
  Speaker 2: it across different demographics.
• 42:46
  Speaker 1: But to do that audit, you actually need to collect
• 42:48
  Speaker 1: vast amounts of demographic data on all of your users,
• 42:52
  Speaker 1: which is a massive violation of their privacy.
• 42:54
  Speaker 2: Exactly. And if you don't collect the data, you can't
• 42:57
  Speaker 2: prove the system is fair.
• 42:58
  Speaker 1: So you're stuck.
• 43:00
  Speaker 2: Well, there are advanced techniques trying to bridge this gap.
• 43:04
  Speaker 2: The text mentions things like federated learning and differential privacy.
• 43:08
  Speaker 1: Okay, let's break those down because they sound like intense jargon.
• 43:12
  Speaker 1: What actually is federated learning?
• 43:14
  Speaker 2: Think of federated learning like keeping all your personal data
• 43:16
  Speaker 2: locked on your own phone.
• 43:18
  Speaker 1: Okay.
• 43:18
  Speaker 2: In the past, companies would suck all your private data
• 43:21
  Speaker 2: up into their central server to train the AI.
• 43:24
  Speaker 1: Which is terrible for privacy.
• 43:26
  Speaker 2: Exactly. With federated learning, the central server sends a copy
• 43:30
  Speaker 2: of the AI model down to your phone. The model
• 43:33
  Speaker 2: learns from your private data locally, gets a little smarter,
• 43:37
  Speaker 2: and then it only sends the updates back to the
• 43:39
  Speaker 2: central server.
• 43:39
  Speaker 1: Wait, so just the new math, not my actual photos
• 43:42
  Speaker 1: or texts?
• 43:43
  Speaker 2: Exactly, just the mathematical updates.
• 43:45
  Speaker 1: Okay, that makes sense. And differential privacy?
• 43:48
  Speaker 2: Differential privacy is basically a technique of adding mathematical static
• 43:53
  Speaker 2: or noise to a data set. Adding noise? If a
• 43:56
  Speaker 2: hacker or even an auditor looks at the data, The
• 44:00
  Speaker 2: noise obscures whether any specific individual person's data is actually
• 44:04
  Speaker 2: in there.
• 44:05
  Speaker 1: But if you zoom out, the overall crowd patterns and
• 44:08
  Speaker 1: statistics remain intact.
• 44:09
  Speaker 2: Exactly.
• 44:10
  Speaker 1: That sounds like a perfect solution.
• 44:11
  Speaker 2: It's clever, but it's a tradeoff. Adding that mathematical noise
• 44:15
  Speaker 2: or keeping data locked on local devices often reduces the
• 44:18
  Speaker 2: overall accuracy of the model.
• 44:20
  Speaker 1: Oh, because it's blurry.
• 44:21
  Speaker 2: Right. It also makes it much harder to run those
• 44:23
  Speaker 2: rigorous audits for fairness we just talked about.
• 44:26
  Speaker 1: Because you can't see the individuals clearly.
• 44:28
  Speaker 2: There is no single optimization target that solves everything. You
• 44:33
  Speaker 2: cannot perfectly maximize privacy, fairness, accuracy, and efficiency all at
• 44:38
  Speaker 2: the exact same time.
• 44:39
  Speaker 1: Responsible AI is about managing these tradeoffs out in the
• 44:42
  Speaker 1: open deliberately instead of pretending they don't exist.
• 44:45
  Speaker 2: And we need to get good at managing these tradeoffs fast.
• 44:48
  Speaker 1: Why fast?
• 44:49
  Speaker 2: Because the text warns about the very near future, the
• 44:52
  Speaker 2: rise of autonomous agents.
• 44:53
  Speaker 1: Agents.
• 44:54
  Speaker 2: Right now, AI mostly makes single, isolated predictions. It recommends
• 44:58
  Speaker 2: a movie or it flags an email as spam or
• 45:01
  Speaker 2: it approves a loan.
• 45:02
  Speaker 1: One action, one result.
• 45:03
  Speaker 2: But we are moving rapidly toward AI agents that make
• 45:06
  Speaker 2: sequences of decisions.
• 45:08
  Speaker 1: Sequences.
• 45:08
  Speaker 2: An AI agent that interacts with a customer, negotiates a schedule,
• 45:12
  Speaker 2: moves money around between accounts, and allocates physical resources all
• 45:16
  Speaker 2: entirely on its own.
• 45:17
  Speaker 1: Without checking in with a human at every single step.
• 45:20
  Speaker 2: Exactly. And if there is a subtle hidden bias in
• 45:23
  Speaker 2: that autonomous system, it can propagate through 10 different actions,
• 45:26
  Speaker 2: creating a massive cascade of unfairness before a human ever
• 45:29
  Speaker 2: even notices something went wrong.
• 45:31
  Speaker 1: This is exactly why risk must dictate oversight. The level
• 45:35
  Speaker 1: of governance and human friction has to match the potential harm.
• 45:38
  Speaker 2: Absolutely. An algorithm that recommends a pop song to you
• 45:41
  Speaker 2: on a Tuesday afternoon does not need the same rigorous
• 45:45
  Speaker 2: legislative guardrails as an algorithm that helps a judge determine
• 45:48
  Speaker 2: a prison sentence or a doctor diagnose a terminal disease.
• 45:52
  Speaker 1: Automation is not automatically beneficial. The text urges organizations to
• 45:57
  Speaker 1: start by asking the most fundamental, courageous question.
• 46:00
  Speaker 2: Should AI be used for this particular decision at all?
• 46:03
  Speaker 1: Exactly. Some things require human values, human empathy, and human context,
• 46:08
  Speaker 1: and fully automating them is simply inappropriate, no matter how
• 46:11
  Speaker 1: good the math is.
• 46:12
  Speaker 2: So if we synthesize everything we've unpacked today, from historical
• 46:15
  Speaker 2: mirrors to proxy variables to impossible math, The core takeaway
• 46:19
  Speaker 2: is this.
• 46:20
  Speaker 1: Let's hear it.
• 46:21
  Speaker 2: Automation does not remove human judgment. That is a myth.
• 46:25
  Speaker 1: What does it do, then?
• 46:26
  Speaker 2: What it actually does is move human judgment to an earlier,
• 46:29
  Speaker 2: often completely invisible stage of the system.
• 46:32
  Speaker 1: Humans still decide what data to collect.
• 46:34
  Speaker 2: Humans decide the mathematical architecture.
• 46:36
  Speaker 1: Humans define the thresholds, and humans decide the metrics of success.
• 46:41
  Speaker 1: The final decision might be printed out by a machine,
• 46:43
  Speaker 1: but the system surrounding that decision is profoundly, inescapably human.
• 46:48
  Speaker 2: Which means an algorithmic prediction is not a certainty.
• 46:52
  Speaker 1: A probability generated by software is not your destiny.
• 46:55
  Speaker 2: And historical numerical data is not neutral just because it's
• 46:58
  Speaker 2: neatly formatted in a spreadsheet.
• 47:00
  Speaker 1: A-literacy is no longer just the niche topic for computer
• 47:03
  Speaker 1: science majors in Silicon Valley. It is a mandatory life
• 47:07
  Speaker 1: skill for every single citizen in the modern world.
• 47:10
  Speaker 2: You have to know how to question the machine.
• 47:11
  Speaker 1: Because the machine is increasingly deciding your future.
• 47:16
  Speaker 2: And that leaves us with one final provocative thought to
• 47:19
  Speaker 2: mull over, building on everything we've learned from this text.
• 47:23
  Speaker 1: If algorithms flawlessly optimize exactly what we tell them to measure,
• 47:27
  Speaker 1: and if we physically cannot mathematically define human fairness.
• 47:30
  Speaker 2: Will the future of our society require us to pass
• 47:33
  Speaker 2: laws mandating something like algorithmic nutrition labels?
• 47:37
  Speaker 1: I think it's a powerful and necessary concept. Think about it.
• 47:41
  Speaker 1: Just as we legally require food companies to print the
• 47:43
  Speaker 1: exact ingredients, the calories, and the nutritional trade-offs on the
• 47:47
  Speaker 1: side of a cereal box.
• 47:48
  Speaker 2: Everyday citizens might legally need to be able to read
• 47:51
  Speaker 2: and audit the specific objective functions of every digital tool
• 47:55
  Speaker 2: they interact with.
• 47:57
  Speaker 1: What is this app actually optimizing for? What mathematical trade-offs
• 48:01
  Speaker 1: did the developers accept behind closed doors? You shouldn't have
• 48:04
  Speaker 1: to be a coder to read the ingredients of the
• 48:06
  Speaker 1: AI that is judging your life.
• 48:08
  Speaker 2: We absolutely demand to know what we put into our
• 48:11
  Speaker 2: biological bodies. It stands to reason we should demand to
• 48:13
  Speaker 2: know what mathematical processes are analyzing and shaping our lives.
• 48:17
  Speaker 1: So the next time you are rejected for that job
• 48:20
  Speaker 1: or that loan or that medical screening.
• 48:22
  Speaker 2: And you hear that murky, unarguable phrase, the algorithm made
• 48:26
  Speaker 2: the decision.
• 48:27
  Speaker 1: Remember that the algorithm was built by people. The data
• 48:30
  Speaker 1: was biased by history. And the decision is only as
• 48:33
  Speaker 1: objective as the flawed human society it mirrors. Thank you
• 48:37
  Speaker 1: so much for joining us on this deep dive. Stay curious,
• 48:40
  Speaker 1: stay skeptical, and always question the objective decisions in your
• 48:43
  Speaker 1: own life.