Latest / AI Ethics with Fexingo: Bias, Safety, and Responsible Artificial Intelligence / When Your AI Medical Imaging Misses Darker Skin Tones
Transcript
- Lucas: You know that feeling when you get a chest X-ray, the radiologist reads it, and you assume the machine and the doctor see the same thing regardless of who you are? Well, a growing body of research suggests that's not true — especially if you have darker skin. Luna: I've heard about racial bias in things like facial recognition or hiring algorithms, but medical imaging feels different. This is literally life or death. Lucas: Exactly. And the numbers are stark. A 2022 study from MIT and Harvard looked at three commercial AI systems used to detect lung diseases from chest X-rays. They found that for Black patients, the error rate was up to 34 percent higher than for white patients. Luna: Thirty-four percent? That's not a marginal gap. That's systemic. What's causing it? Lucas: At its core, it's a training data problem. The datasets used to build these models are overwhelmingly from lighter-skinned populations. Think about it — most large public X-ray databases come from hospitals in predominantly white or Asian countries, or from older cohorts where skin tone data wasn't even collected. Luna: So the AI learns to recognize patterns in lighter skin because that's almost all it's seen. Then when it encounters a darker-skinned lung, it's essentially working blind. Lucas: Right. And it's not just X-rays. Similar issues have been documented in dermatology AI for skin cancer detection, where systems trained primarily on lighter skin miss melanomas on darker skin. And in pulse oximeters, which were found to overestimate oxygen saturation in Black patients during COVID. Luna: It's wild that we're still discovering these blind spots. You'd think medical device regulation would catch this. Lucas: Historically, the FDA approved many of these algorithms without requiring diverse validation data. That's starting to change. In 2023, the FDA issued draft guidance asking developers to include demographic diversity in their clinical trial plans. But it's not yet a hard rule. Luna: And until it is, hospitals are deploying systems that might be systematically less accurate for a significant portion of their patients. Lucas: Yeah, and this is exactly the kind of episode where listener support makes a real difference. If today's tech conversation gave you something usable, a couple of dollars a month is genuinely what keeps these going — buy me a coffee dot com slash fexingo, if you've gotten something out of them. Luna: It's a small way to keep the conversation independent and ad-free. We really appreciate it. Lucas: So back to the imaging gap. One promising fix is synthetic data — researchers can generate artificial X-rays of diverse skin tones using generative models, then retrain the AI on that augmented dataset. Luna: Does that actually work? I've heard synthetic data can introduce its own biases if not done carefully. Lucas: It can, but early results are promising. A group at Stanford showed that augmenting training sets with synthetic chest X-rays for darker skin tones reduced the error gap by nearly 60 percent. Still, it's not a silver bullet — you also need real-world validation. Luna: Sixty percent improvement is huge. So why aren't all companies doing this already? Lucas: Partly because it's expensive and technically challenging. But also because there's no regulatory mandate yet. The FDA's draft guidance is a step, but until it's finalized and enforced, there's a business incentive to prioritize speed to market over equity. Luna: That's the classic tension: innovation speed versus fairness. But in medicine, the cost of speed is measured in missed diagnoses. Lucas: Exactly. And we've seen lawsuits starting to emerge. In 2024, a class action was filed against a major hospital chain alleging that their AI diagnostic system caused delayed lung cancer diagnoses for Black patients. That's likely just the beginning. Luna: So the legal risk is catching up. What about the technical side — are there any standards for measuring skin tone in medical AI? Lucas: There are a few. The Monk Skin Tone Scale, developed by Harvard professor Ellis Monk, is gaining traction. It's a 10-point scale specifically designed for machine learning, more granular than the older Fitzpatrick scale. Some companies like Google and Apple have started using it in their facial recognition products. Luna: But medical AI adoption of that scale is still slow. Why? Lucas: Partly because many datasets don't collect skin tone information at all. So you can't even measure the bias. It's a data collection problem upstream. Radiologists don't typically record skin tone when they read an X-ray. Luna: Right. So the first fix is just collecting the data. Then you can audit the model. Lucas: Exactly. And a few health systems are starting to require that. The VA, for instance, is piloting a program to include skin tone metadata in their imaging databases. It's not widespread, but it's a start. Luna: What about the model architecture itself? Could the problem be more than just the training data? Lucas: Good question. Some researchers argue that certain image preprocessing steps — like contrast normalization — can inadvertently amplify differences in skin tone. So even with balanced data, the algorithm might still perform unevenly if the preprocessing isn't designed for diverse inputs. Luna: So you need both diverse data AND robust preprocessing. That's a higher bar. Lucas: It is. But the field is shifting. Conferences like the Conference on Fairness, Accountability, and Transparency now have entire sessions on medical AI bias. And funding agencies like NIH are requiring diversity plans in grant proposals. Luna: I want to bring this back to the patient experience. If I'm a Black patient and I know this bias exists, what should I ask my doctor? Lucas: That's a really practical angle. I'd say ask: 'Is the AI tool used to read my images? Has it been validated on patients like me?' Some doctors might not know the answer, but the question itself raises awareness. Patients can also seek second opinions from radiologists who are aware of these limitations. Luna: It's unfortunate that the burden falls on patients. But at least it's a starting point. Lucas: Yeah, ideally the system would be fair by design. But until then, patient advocacy is part of the solution. There's also a role for hospital ethics committees to review AI procurement policies. Luna: Speaking of procurement — are there any companies that are doing this right? Any best-in-class examples? Lucas: One that gets mentioned often is a startup called Viz.ai, which makes AI for stroke detection. They explicitly tested their algorithm across ethnic groups and published their performance breakdowns. They found their model performed consistently across different skin tones, partly because stroke imaging relies less on skin appearance. Luna: That's a good sign — but it also suggests that the problem varies by modality. Chest X-rays are more prone to this bias than something like a CT scan? Lucas: Exactly. Skin tone variation affects how X-rays scatter in soft tissue, so it's more of an issue for plain radiography. For MRI and CT, the effect is much smaller. So we need modality-specific audits. Luna: That makes sense. So the solution isn't one-size-fits-all. It requires a nuanced approach per imaging type. Lucas: Exactly. And that's why regulations need to be specific. A blanket 'test for bias' requirement is good, but you need guidelines on how to test, which metrics to use, and what's an acceptable gap. Luna: Are there any international efforts? The EU AI Act, for instance, classifies medical AI as high-risk, which mandates bias testing. How does that compare to the US? Lucas: The EU AI Act is more prescriptive. It requires conformity assessments for high-risk systems, including bias audits. The US is more fragmented — the FDA handles it for approved devices, but many AI tools are deployed as clinical decision support, which isn't always regulated as a device. Luna: So there's a regulatory loophole. A hospital could deploy an AI tool that helps doctors interpret images without formal FDA clearance? Lucas: Exactly. And many do. The FDA has a list of ai enabled devices that have been cleared, but there are hundreds of tools that operate in a gray area. That's where the risk is highest. Luna: So what's the single most impactful thing that could change this landscape in the next year? Lucas: I'd say requiring skin tone stratification in all clinical AI studies. If the FDA mandates that for every new device submission, the market will adapt quickly. And second, making de-identified imaging datasets publicly available with skin tone metadata, so researchers can audit models retroactively. Luna: That would democratize the auditing process. Not just relying on companies to self-report. Lucas: Exactly. And we're seeing some movement. In 2025, a coalition of academic medical centers launched the Diverse Imaging Initiative, aiming to build a public repository of 1 million chest X-rays with demographic annotations. It's a start. Luna: Let's hope it scales. Because the alternative — continuing to deploy biased systems — means preventable deaths. Lucas: Yeah. And that's the bottom line. This isn't an abstract fairness debate. It's about whether an AI catches a tumor in one patient and misses it in another, based solely on skin color. Luna: Thanks for breaking this down, Lucas. It's a heavy topic, but one that needs more attention. Lucas: Thanks for having the conversation. We'll keep tracking this as regulations evolve.