Latest / Minds, Bodies, and Terawatts / California Just Made It Illegal for Software to Fire You on Its Own. The Fine Is $500, the Key Word Is Undefined, and Uber and Lyft Drivers Were Written Out

Transcript

California Just Made It Illegal for Software to Fire You on Its Own. The Fine Is $500, the Key Word Is Undefined, and Uber and Lyft Drivers Were Written Out · Minds, Bodies, and Terawatts

• 0:00
  [Host] This is Minds, Bodies, and Terawatts, the show where the pink slip now needs a second signature. CNBC reported that on Wednesday, September thirtieth, California Governor Gavin Newsom signed the No Robo Bosses Act, the first law in the country that says software can't fire you by itself.
• 0:16
  [Guest] Fire you or discipline you. Starting next July, a California employer can't rely solely on an automated system for either one, and the law covers plain scoring and ranking software, not only the products sold as AI.
• 0:29
  [Host] My boss is a person, though. Is anybody really getting fired by software?
• 0:33
  [Guest] More of the job runs through software than you'd think. In an OECD survey of wealthy countries, ninety percent of American managers said their company uses at least one tool to instruct, monitor or evaluate workers.
• 0:45
  [Host] Monitoring me isn't firing me.
• 0:47
  [Guest] No, but the score is where a firing starts. Former Meta employees sued this summer, saying an AI-assisted system ranked people for layoffs and landed hardest on those who had taken medical or family leave. Meta denies it.
• 1:00
  [Host] So now a human has to make the call. That sounds like the fix.
• 1:05
  [Guest] Look at what the human is asked to do. When the employer relies primarily on the system's output, it has to direct a person to corroborate it, and if that person can't back it up, the company can't use it.
• 1:18
  [Host] Corroborate it with what, exactly?
• 1:20
  [Guest] The bill says the data the system itself used, or other material like manager evaluations and your personnel file. So the second signature can come from someone reading the same file the machine read.
• 1:32
  [Host] Then is that a second opinion, or only a second signature?
• 1:35
  [Guest] That hangs on a word the law never defines. Everything kicks in when the employer primarily relies on the system, and nothing in the text says when a tool stops informing a decision and starts driving it.
• 1:48
  [Host] A tech industry group made that exact complaint when it asked Newsom for a veto, so why should it worry me?
• 1:55
  [Guest] Because the gap cuts against you too. A company that says the score was one input among several never triggers the review, and never owes you the notice.
• 2:05
  [Host] What notice?
• 2:05
  [Guest] This is the part you can actually use. If software was the primary basis, you're told in writing, at the moment you get the news, that a system was used and that a human reviewed it, with a person you can contact. You can also demand a description of your own data that went in.
• 2:24
  [Host] So I'd finally see what the machine had on me. And if the company skips all of it?
• 2:30
  [Guest] The fine is five hundred dollars per violation, and the labor commissioner or a prosecutor has to take the case, because you can't sue under this law yourself.
• 2:40
  [Host] Say one does take it. Does "the score was just one input" still get the company off?
• 2:46
  [Guest] That gets harder, because once it's shown the software was used, the burden flips and the company has to prove it wasn't the main basis.
• 2:54
  [Host] Newsom vetoed an earlier version a year ago, so what changed?
• 2:58
  [Guest] The bill shrank. The version he vetoed made employers tell you in advance when this software was in use, and earlier drafts of this one reached gig workers. Both are gone from what he signed, the gig piece after heavy pushback from Uber and Lyft.
• 3:13
  [Host] Wait, the drivers are out? An app cutting off a driver is the robo boss everybody pictures.
• 3:19
  [Guest] The law covers employees, and that is where it stops. A driver cut off by an app gets no review and no notice.
• 3:27
  [Host] Back to that signature, then. Say I'm fired in California next summer, how do I tell whether the human actually decided?
• 3:34
  [Guest] You'd want to know what the human was shown. A firing gets settled upstream, in what went into the file and what the score said before anyone looked. A name at the bottom tells you who signed.
• 3:47
  [Host] Won't every company just make sure there's a name at the bottom?
• 3:51
  [Guest] Once "a human reviewed it" is the legal test, reviews get produced. What would make one real is a record of what the reviewer looked at and how often they overrule the machine, and the law doesn't require either. You get to see what the software was fed about you, and nothing about what the reviewer did with it.
• 4:11
  [Host] What if I don't work in California?
• 4:13
  [Guest] Then you get none of it. A federal bill with the same name went nowhere in the Senate, and a handful of other states have versions waiting.
• 4:23
  [Host] And if I do, and it happens to me?
• 4:26
  [Guest] Ask in writing whether a system was the primary basis for the decision, and ask for your data.
• 4:32
  [Host] By then I'm already out the door. What's a piece of paper worth?
• 4:36
  [Guest] It's your first look at the file. Patrick Deglon's rule in Unscarcity is no decision in the dark, and that has to include the one behind the second signature. For more on who really decides when a human signs off on a machine, head to unscarcity dot ai. The book is on Amazon, the audiobook is on Spotify. This has been Minds, Bodies, and Terawatts.