Slicing Through Bias: Explaining Performance Gaps in Medical Image Analysis using Slice Discovery Methods

TL;DR

This paper introduces a novel Slice Discovery Method to identify and explain performance disparities in medical image analysis, revealing sex-based shortcut learning as a key factor affecting model fairness.

Contribution

The paper presents a new SDM technique and demonstrates its effectiveness in uncovering and explaining performance gaps in chest X-ray classification tasks.

Findings

Identified sex-based performance disparities in chest X-ray models.

Revealed shortcut features like chest drains and ECG wires influence model decisions.

Showed SDMs can generate hypotheses about causes of performance gaps.

Abstract

Machine learning models have achieved high overall accuracy in medical image analysis. However, performance disparities on specific patient groups pose challenges to their clinical utility, safety, and fairness. This can affect known patient groups - such as those based on sex, age, or disease subtype - as well as previously unknown and unlabeled groups. Furthermore, the root cause of such observed performance disparities is often challenging to uncover, hindering mitigation efforts. In this paper, to address these issues, we leverage Slice Discovery Methods (SDMs) to identify interpretable underperforming subsets of data and formulate hypotheses regarding the cause of observed performance disparities. We introduce a novel SDM and apply it in a case study on the classification of pneumothorax and atelectasis from chest x-rays. Our study demonstrates the effectiveness of SDMs in hypothesis formulation and yields an explanation of previously observed but unexplained performance disparities between male and female patients in widely used chest X-ray datasets and models. Our findings indicate shortcut learning in both classification tasks, through the presence of chest drains and ECG wires, respectively. Sex-based differences in the prevalence of these shortcut features appear to cause the observed classification performance gap, representing a previously underappreciated interaction between shortcut learning and model fairness analyses.