Reading Race: AI Recognises Patient's Racial Identity In Medical Images
Imon Banerjee, Ananth Reddy Bhimireddy, John L. Burns, Leo Anthony, Celi, Li-Ching Chen, Ramon Correa, Natalie Dullerud, Marzyeh Ghassemi,, Shih-Cheng Huang, Po-Chih Kuo, Matthew P Lungren, Lyle Palmer, Brandon J, Price, Saptarshi Purkayastha, Ayis Pyrros, Luke Oakden-Rayner

TL;DR
This study demonstrates that AI models can reliably predict patients' race from medical images across various modalities, raising concerns about potential biases and misuse in clinical settings.
Contribution
The paper reveals that deep learning models can accurately detect race from medical images, even when traditional proxies are absent, highlighting a significant challenge for fairness in AI-driven healthcare.
Findings
AI models predict race with high accuracy across modalities
Detection persists despite image corruption and noise
Race prediction is not due to trivial proxies or disease distribution
Abstract
Background: In medical imaging, prior studies have demonstrated disparate AI performance by race, yet there is no known correlation for race on medical imaging that would be obvious to the human expert interpreting the images. Methods: Using private and public datasets we evaluate: A) performance quantification of deep learning models to detect race from medical images, including the ability of these models to generalize to external environments and across multiple imaging modalities, B) assessment of possible confounding anatomic and phenotype population features, such as disease distribution and body habitus as predictors of race, and C) investigation into the underlying mechanism by which AI models can recognize race. Findings: Standard deep learning models can be trained to predict race from medical images with high performance across multiple imaging modalities. Our findings…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
[ML News] MMO Game destroys GPUs | OpenAI quits Robotics | Today w/ guest host Sanyam Bhutani· youtube
Taxonomy
TopicsArtificial Intelligence in Healthcare and Education · Radiology practices and education · Radiomics and Machine Learning in Medical Imaging
