Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Exam Proctor Accuses You of Cheating
Transcript
- Lucas: If today's tech conversation gave you something usable, that's exactly why we do this. And it's also why a small number of listeners chip in monthly through buy me a coffee dot com slash fexingo. Luna: Yeah, that direct support is what keeps the show independent and ad-free. No sponsors, no pressure to cover a product. Lucas: Right. So if you find yourself reaching for these episodes in your feed regularly, you know where to find that page. Okay — let's get into today's case. Lucas: Remote proctoring software — the stuff that watches you take an exam from your dorm room — has become a permanent fixture at a lot of universities. And it's generating a quiet crisis of false accusations. Luna: You're talking about the automated systems that flag you for looking away from the screen, or moving your mouth, or even just pausing too long? Lucas: Exactly. Let's anchor on one specific case that got some attention last year. A student at the University of North Carolina — let's call her Sarah — was taking a timed online exam. The proctoring software, a well-known product from a company called Honorlock, flagged her for 'suspicious eye movement.' Her exam was invalidated. She failed the course. Luna: And she had no chance to explain herself? No human review? Lucas: Not initially. The university's policy was that the ai generated flag was sufficient grounds for an automatic zero. It took her three months of appeals, involving the department chair and the dean, before they agreed to let her retake the exam. And the entire time, the university hadn't even watched the flagged footage — it was just an algorithm report. Luna: So the AI's word was basically gospel until she pushed back hard enough. That's a reversal of the presumption of innocence. Lucas: It is. And the data on false positives in these systems is troubling. A 2023 study from MIT and the University of Michigan looked at four major proctoring platforms — ProctorU, Honorlock, ExamSoft, and Respondus. They found false positive rates ranging from 3 percent to 15 percent depending on the system and the type of behavior flagged. Luna: Fifteen percent is huge. That means one in seven students could be falsely accused in a given exam. Lucas: And the consequences are real. Some students have had their degrees delayed, lost scholarships, or been placed on academic probation. The report also found that the systems were more likely to flag students with darker skin tones — particularly for gaze detection, because the algorithms are trained primarily on lighter-skinned faces. Luna: So you've got a technology that's not only unreliable, but also racially biased. And universities are deploying it at scale without, it sounds like, proper oversight. Lucas: That's the core issue. Let's talk about why the false positives happen. These systems typically use computer vision to track eye movement, head position, and mouth movement. They look for patterns that correlate with cheating — like reading off a second screen or looking at notes off-camera. But the problem is that human behavior is incredibly varied. Luna: Right — some people naturally look up when thinking, or they mouth words to themselves when reading a tough question. Lucas: Exactly. The systems don't distinguish between a nervous tic and actual cheating. And the training data for these models is often collected in controlled environments — not real exam conditions. So the algorithms learn a narrow version of 'normal' behavior. Luna: And what about the privacy angle? These systems are recording video of students in their homes. That video is stored on company servers. Lucas: That's another layer. In Sarah's case, the university required students to install a browser extension that had full access to their browsing activity during the exam. That's a significant privacy intrusion, especially for students who may not have a private room and are forced to take exams in shared living spaces. Luna: So there's surveillance creep. The technology that was supposed to maintain academic integrity is actually creating new forms of inequity. Lucas: And the irony is that there's no good evidence these systems actually reduce cheating. A 2022 meta-analysis by the Journal of Computer Assisted Learning reviewed 15 studies on automated proctoring and found no statistically significant reduction in cheating rates compared to in-person proctoring. Meanwhile, the false accusation rate was consistently non-trivial. Luna: So we're imposing a flawed system on students for no measurable benefit. That's a policy failure. Lucas: It is. And the question is — what should universities do instead? Some schools have moved to open-book exams that emphasize critical thinking over memorization. Others use randomized question pools and time limits that make cheating less effective. And a few have simply returned to in-person exams. Luna: But those options aren't available for every institution, especially large online programs. Lucas: True. But at minimum, schools should have a clear appeals process with human review before any academic penalty is applied. A handful of states — including New York and California — are considering legislation that would require exactly that. The bills would also mandate transparency about the false positive rates of the specific software used. Luna: That seems like a basic accountability measure. If a university is going to use AI to police students, they should know how often it gets it wrong. Lucas: And they should share that data publicly. Right now, most universities keep their proctoring contracts confidential. Students don't know what they're agreeing to when they download the software. Luna: So the next time someone tells you AI is making education more fair... you might want to ask whose fairness we're talking about. Lucas: Exactly. The technology is here to stay in some form, but the ethics of how it's deployed are still very much in play. Quick honest thing — making episodes like this, digging into specific cases, takes time and research. A handful of listeners chip in monthly through buy me a coffee dot com slash fexingo, and that's literally what funds making this many of these. Luna: It's true. No ads, no sponsors, just listener support. It keeps us free to follow the story where it leads. Lucas: And we appreciate every bit of that support. Alright — back to the topic. One thing that struck me in the UNC case is that the professor himself later admitted he hadn't watched the flagged footage. He just relied on the software's report. Luna: So even the human in the loop wasn't really in the loop. Lucas: Right. And that's a recurring pattern across many AI systems — the 'human review' becomes a rubber stamp. We've seen it in hiring, in loan approvals, and now in education. Luna: What's the fix at the institutional level? Should universities be required to retain logs and footage for a set period, so that appeals can actually be investigated? Lucas: That would be one step. Another is to require that any flag be reviewed by a human before any action is taken. And that human should be trained on the system's limitations. There are also technical fixes — like building systems that give confidence scores rather than binary 'cheating' flags. A probability, not a verdict. Luna: That would at least force the human reviewer to think critically about the evidence. Lucas: Exactly. And it would reduce the stigma of being accused. Because right now, even if you're exonerated, the accusation itself can create a shadow — especially if the AI's report is shared with faculty or administrators. Luna: So the ethical challenge here isn't just about the technology. It's about the systems of power that adopt it without safeguards. Lucas: That's the thread running through so many of these episodes. AI is a tool, but the context around it determines whether it helps or harms. And right now, in proctoring, the harm is real and the benefits are unproven. Luna: Something to keep in mind the next time you see a 'secure exam' notice. Lucas: Absolutely. And that's where we'll leave it today. Thanks for listening.