Bias in Machine Learning Models Can Be Significantly Mitigated by Careful Training: Evidence from Neuroimaging Studies

TL;DR

This study demonstrates that with careful training, machine learning models in neuroimaging can achieve unbiased, accurate predictions across diverse demographic groups and conditions, challenging assumptions about inherent bias.

Contribution

The paper provides evidence that properly trained models using multi-source data can generalize well and reduce bias in neuroimaging diagnostics for brain diseases.

Findings

Models trained with multi-source data are less biased across demographic groups.

Proper training can lead to high AUC and fairness in diverse subgroups.

Additional demographic and clinical features can improve model performance and fairness.

Abstract

Despite the great promise that machine learning has offered in many fields of medicine, it has also raised concerns about potential biases and poor generalization across genders, age distributions, races and ethnicities, hospitals, and data acquisition equipment and protocols. In the current study, and in the context of three brain diseases, we provide evidence which suggests that when properly trained, machine learning models can generalize well across diverse conditions and do not necessarily suffer from bias. Specifically, by using multi-study magnetic resonance imaging consortia for diagnosing Alzheimer's disease, schizophrenia, and autism spectrum disorder, we find that well-trained models have a high area-under-the-curve (AUC) on subjects across different subgroups pertaining to attributes such as gender, age, racial groups, and different clinical studies and are unbiased under multiple fairness metrics such as demographic parity difference, equalized odds difference, equal opportunity difference etc. We find that models that incorporate multi-source data from demographic, clinical, genetic factors and cognitive scores are also unbiased. These models have better predictive AUC across subgroups than those trained only with imaging features but there are also situations when these additional features do not help.