Latest / Privacy Please / S7, E269 - You're the Teacher Now: How Companies Are Using Your Data to Build AI That Replaces You

Transcript

S7, E269 - You're the Teacher Now: How Companies Are Using Your Data to Build AI That Replaces You · Privacy Please

• 0:00
  SPEAKER_00: I think we've all heard the phrase.
• 0:02
  SPEAKER_00: You're the product.
• 0:04
  SPEAKER_00: What a service is free.
• 0:05
  SPEAKER_00: Your data is what's being sold.
• 0:10
  SPEAKER_00: That was the deal.
• 0:11
  SPEAKER_00: A little uncomfortable, but at least we understood it.
• 0:16
  SPEAKER_00: Here's what nobody's told you about the new deal.
• 0:20
  SPEAKER_00: You're not just the product anymore.
• 0:22
  SPEAKER_00: You're the teacher.
• 0:23
  SPEAKER_00: Your emails, your chats, your support tickets, your work decisions, your daily patterns.
• 0:28
  SPEAKER_00: Companies are feeding into all of it.
• 0:32
  SPEAKER_00: They're feeding all of it into AI systems right now.
• 0:36
  SPEAKER_00: Systems that, in a lot of cases, are being built specifically to do what you do.
• 0:42
  SPEAKER_00: GitHub is just the latest company to make it official.
• 0:45
  SPEAKER_00: They're using your code, your prompts, your inputs and outputs to train their AI models.
• 0:52
  SPEAKER_00: Opted in by default.
• 0:54
  SPEAKER_00: Four menus deep and settings if you want out.
• 0:58
  SPEAKER_00: That's just the one that got caught.
• 1:00
  SPEAKER_00: We need to talk about the rest of it.
• 1:22
  SPEAKER_00: And today we're talking about something that affects every single person listening right now, probably, most likely, 99% of you.
• 1:29
  SPEAKER_00: Um, most of you, all of you, I'm pretty sure of it.
• 1:33
  SPEAKER_00: Uh, and if you can't tell, I'm dealing with some kind of sickness or I'm losing my voice.
• 1:39
  SPEAKER_00: I'm not really sure, but apologies if my voice cracks, but uh the train keeps moving.
• 1:47
  SPEAKER_00: So AI systems need data the way that factories need raw materials, right?
• 1:53
  SPEAKER_00: So the more the better.
• 1:55
  SPEAKER_00: And unlike oil, data is everywhere.
• 1:59
  SPEAKER_00: It's being generated constantly by hundreds of millions of people just going about their day.
• 2:05
  SPEAKER_00: For years, the model was simple: scrape the internet, use public data, but regulators and courts have started pushing back on all of that.
• 2:16
  SPEAKER_00: Copyright laws, GDPR enforcement, platform access restrictions.
• 2:21
  SPEAKER_00: So companies turn to the next best thing: the data they already have, data their customers gave them, data their employees generate every single day.
• 2:32
  SPEAKER_00: There's a legal term for what happens when data collected for one purpose gets quietly repurposed for something else.
• 2:40
  SPEAKER_00: Function creep.
• 2:41
  SPEAKER_00: You signed up for a productivity app, your documents are now training a language model, you used a customer service chatbot, those transcripts are now teaching the next version of the chatbot how to sound more human.
• 2:54
  SPEAKER_00: You uploaded photos to the cloud backup, those images are now part of an image recognition data set.
• 3:00
  SPEAKER_00: It just keeps going.
• 3:01
  SPEAKER_00: The original consent covered the original purpose.
• 3:05
  SPEAKER_00: It didn't cover this.
• 3:07
  SPEAKER_00: And in most cases, it nobody asked for it.
• 3:12
  SPEAKER_00: The most recent example is GitHub.
• 3:15
  SPEAKER_00: Starting April 24th of 2026, Microsoft's code platform is going to use customer interaction data, specifically your inputs, outputs, code snippets, and context from Copilot to train its AI models.
• 3:32
  SPEAKER_00: The policy applies to Copilot free pro and pro plus users.
• 3:37
  SPEAKER_00: Enterprise and business customers are exempt because their contract's specifically prohibited, which tells you something important.
• 3:45
  SPEAKER_00: The protection exists, they just didn't extend it to everyone.
• 3:50
  SPEAKER_00: You can opt out if GitHub says, but to do it, you have to navigate to settings, find the copilot section, find the privacy heading, disable a toggle that says allow GitHub to use my data for AI model training.
• 4:05
  SPEAKER_00: It's off by default in some regions because European law requires opt-in.
• 4:10
  SPEAKER_00: In the US, it's on by default because the law doesn't require otherwise.
• 4:16
  SPEAKER_00: How that's even a thing in the US boggles my mind to this day.
• 4:22
  SPEAKER_00: But here we are, we're still trucking along, right?
• 4:26
  SPEAKER_00: That isn't on accident.
• 4:27
  SPEAKER_00: It's the architecture of consent in 2026.
• 4:30
  SPEAKER_00: GitHub isn't alone.
• 4:32
  SPEAKER_00: This is the playbook.
• 4:33
  SPEAKER_00: A quiet policy update, a pre-selected toggle, an opt-out buried four menus deep.
• 4:40
  SPEAKER_00: By the time most users notice their data has been feeding the model for months.
• 4:44
  SPEAKER_00: The FTC has been clear on paper.
• 4:47
  SPEAKER_00: Using customer data for secret purposes, including training AI models, can be an unfair or deceptive practice.
• 4:55
  SPEAKER_00: There's a lawsuit right now, Con vs.
• 4:57
  SPEAKER_00: Figma, that's testing exactly this theory.
• 5:00
  SPEAKER_00: Figma allegedly used years of customers' design files to train generative AI tools for assuring users their content wouldn't be repurposed.
• 5:10
  SPEAKER_00: The suit doesn't hinge on copyright, it hinges on consent.
• 5:15
  SPEAKER_00: And that's what makes it dangerous for every company running the same playbook.
• 5:21
  SPEAKER_00: The customer data story is uncomfortable.
• 5:25
  SPEAKER_00: The employee data story is something else entirely.
• 5:29
  SPEAKER_00: Across industries right now, a pattern is emerging.
• 5:33
  SPEAKER_00: Companies are asking employees to document their workflows, label data sets, record their decision-making processes, and workers are increasingly realizing that what they're building is the training set for the AI that replaces them.
• 5:49
  SPEAKER_00: Customer service reps, recording calls, financial analysts documenting how they evaluate risk, legal teams tagging contracts, all of it feeding into systems specifically designed to automate those functions.
• 6:05
  SPEAKER_00: At Amazon, driver data was used to train autonomous routing algorithms.
• 6:10
  SPEAKER_00: At Google, employees spent months training an internal AI called Gemini for sales on client relationship workflows.
• 6:18
  SPEAKER_00: Then the company cut hundreds of ad sale positions.
• 6:21
  SPEAKER_00: Former employees described it to Business Insider as building your own coffin.
• 6:26
  SPEAKER_00: JP Morgan Chase trained a contract analysis tool partly on the work of the legal and analysts whose roles it was designed to streamline.
• 6:38
  SPEAKER_00: Here's the legal reality here.
• 6:40
  SPEAKER_00: Most workers have almost no protection here.
• 6:43
  SPEAKER_00: If your employer asks you to participate in AI training programs, you're likely required to comply.
• 6:49
  SPEAKER_00: Employment lawyers say that documenting everything is the best defense.
• 6:54
  SPEAKER_00: If your employer tells you the AI training won't affect your position and then lays you off six months later, that paper trail matters.
• 7:03
  SPEAKER_00: The power imbalance in employment relationships is also why European regulators are skeptical of employee consent for AI training purposes.
• 7:13
  SPEAKER_00: Consent isn't freely given when you can't say no without risking your job.
• 7:19
  SPEAKER_00: And there's a compounding problem nobody talks about here.
• 7:23
  SPEAKER_00: It's the bias baked in at the source.
• 7:26
  SPEAKER_00: If the training data reflects existing workplace patterns, who got promoted, whose decisions got flagged, whose work got praised, the AI inherits all of it.
• 7:37
  SPEAKER_00: Not as a bug, as a feature.
• 7:40
  SPEAKER_00: Because that's what it learned from.
• 7:42
  SPEAKER_00: Here's the part that should really unsettle you the most.
• 7:47
  SPEAKER_00: Privacy law gives you the right to have your data deleted.
• 7:50
  SPEAKER_00: Under GDPR, CCPA, and a growing list of state laws, you can ask a company to remove your information, in theory.
• 7:59
  SPEAKER_00: In practice, AI training breaks all of that.
• 8:05
  SPEAKER_00: When a company deletes your record from their database, the raw data is gone.
• 8:10
  SPEAKER_00: But the model that learned from it, that's still running.
• 8:13
  SPEAKER_00: Your patterns, your behavior, your decisions, they've been absorbed into the weights of a neural network.
• 8:20
  SPEAKER_00: You can't delete yourself from a trained model without retaining the model from scratch.
• 8:26
  SPEAKER_00: Most companies aren't doing that.
• 8:28
  SPEAKER_00: Legal authorities are starting to say the model itself constitutes derived personal data when it was trained on individual records.
• 8:37
  SPEAKER_00: That's an enormous problem for every AI product built on customer data without rigorous consent processes.
• 8:45
  SPEAKER_00: And it means deletion rights.
• 8:47
  SPEAKER_00: One of the most important protections consumers have may be functionally unenforceable in the AI context right now.
• 8:57
  SPEAKER_00: So what do you do about this?
• 9:00
  SPEAKER_00: What do you do with this?
• 9:02
  SPEAKER_00: Audit your tools.
• 9:04
  SPEAKER_00: Any app or platform you use regularly, especially ones using AI features, has a privacy settings page.
• 9:11
  SPEAKER_00: Look for toggles related to improving our services, personalization, or anything that mentions AI model training.
• 9:18
  SPEAKER_00: Assume they're on by default.
• 9:21
  SPEAKER_00: Turn them off.
• 9:22
  SPEAKER_00: For GitHub specifically, settings, co-pilot, privacy, disable.
• 9:29
  SPEAKER_00: Allow GitHub to use my data for AI model training.
• 9:33
  SPEAKER_00: Do it before April 24th.
• 9:35
  SPEAKER_00: If your employer asks you to participate in AI training, document what you've been told about how the data will be used and whether it affects your role.
• 9:44
  SPEAKER_00: Don't rely on verbal reassurances.
• 9:47
  SPEAKER_00: Know your rights.
• 9:48
  SPEAKER_00: If you're in California, Colorado, Virginia, or any of the 17 other states with comprehensive privacy laws, you have data access and deletion rights.
• 9:59
  SPEAKER_00: Use them.
• 10:00
  SPEAKER_00: If deletion from a train model is complicated, the paper trail matters.
• 10:05
  SPEAKER_00: I challenge you to ask the question companies don't want you to ask.
• 10:09
  SPEAKER_00: When you use any AI-powered tool, ask yourself, is my input being used to train this?
• 10:16
  SPEAKER_00: If the answer is unclear, assume it's a yes.
• 10:21
  SPEAKER_00: The phrase function creep is almost too gentle for what's happening.
• 10:26
  SPEAKER_00: Your data is being repurposed for a use you never agreed to, your work is being used to build systems designed to replace you, and the opt-out, when it exists at all, is buried deep and enough that most people will never find it.
• 10:42
  SPEAKER_00: This isn't a future problem.
• 10:44
  SPEAKER_00: GitHub's policy change is coming up on April 24th.
• 10:48
  SPEAKER_00: That's a few weeks away.
• 10:50
  SPEAKER_00: The question we're sitting with isn't whether companies should be allowed to do this.
• 10:56
  SPEAKER_00: It's whether you ever actually agreed to it.
• 10:58
  SPEAKER_00: And if the answer is no, what happens next?
• 11:05
  SPEAKER_00: I'm Cameron Ivey.
• 11:07
  SPEAKER_00: This was Privacy Please, part of the Problem Lounge Network.
• 11:10
  SPEAKER_00: If this hit close to home for you, and it should, share it with someone, please.
• 11:16
  SPEAKER_00: If you share it, it uh helps us spread the show out to more people, which, you know, I think could help a lot of people right now.
• 11:24
  SPEAKER_00: You can find all of our shows, episodes, and more at theproblemlounge.com.
• 11:29
  SPEAKER_00: Subscribe, follow, leave us a review if if you're feeling generous, it would really help us reach more people, and that's that's the main goal here, just to to try to spread good information that is helpful in this space.
• 11:42
  SPEAKER_00: So thank you so much for tuning in if it's your first time and if you've been with us for a long time.
• 11:47
  SPEAKER_00: Thank you so much for the support.
• 11:49
  SPEAKER_00: And uh, I just know that I love doing these.
• 11:52
  SPEAKER_00: So sorry for the voice this week, and uh we'll see you guys on the next one.
• 11:56
  SPEAKER_00: Cameron Ivy, over and out.