Risk of Training Diagnostic Algorithms on Data with Demographic Bias

TL;DR

This paper highlights the importance of considering demographic biases in training diagnostic algorithms, demonstrating how such biases affect performance and proposing adversarial training to mitigate them.

Contribution

It reveals the lack of demographic consideration in medical image analysis papers and introduces an adversarial training method to reduce demographic bias in classifiers.

Findings

Performance varies across demographic subgroups despite balanced datasets.

Adversarial training can produce more equitable diagnostic classifiers.

Ignoring demographics can lead to biased medical predictions.

Abstract

One of the critical challenges in machine learning applications is to have fair predictions. There are numerous recent examples in various domains that convincingly show that algorithms trained with biased datasets can easily lead to erroneous or discriminatory conclusions. This is even more crucial in clinical applications where the predictive algorithms are designed mainly based on a limited or given set of medical images and demographic variables such as age, sex and race are not taken into account. In this work, we conduct a survey of the MICCAI 2018 proceedings to investigate the common practice in medical image analysis applications. Surprisingly, we found that papers focusing on diagnosis rarely describe the demographics of the datasets used, and the diagnosis is purely based on images. In order to highlight the importance of considering the demographics in diagnosis tasks, we used a publicly available dataset of skin lesions. We then demonstrate that a classifier with an overall area under the curve (AUC) of 0.83 has variable performance between 0.76 and 0.91 on subgroups based on age and sex, even though the training set was relatively balanced. Moreover, we show that it is possible to learn unbiased features by explicitly using demographic variables in an adversarial training setup, which leads to balanced scores per subgroups. Finally, we discuss the implications of these results and provide recommendations for further research.