AI automation bias: when human oversight stops working
A military decision support tool hallucinated and flagged Chinese nuclear infrastructure as an active threat, and a human caught it just in time. Was that a success? The position here is no. Oversight only counts if it can keep pace with the system, and volume plus automation bias means reviewers slide into rubber stamping. A model that is wrong 3 to 8 percent of the time, the range Google's AI safety team reports for frontier models, is more dangerous than one that is obviously unreliable.
You leave with three design rules used by operators who get this right: cap reviewer throughput, force…
The skinny
The skinny isn't ready yet — notes appear once the transcript is processed.