Distributionally Robust Optimization and Invariant Representation Learning for Addressing Subgroup Underrepresentation: Mechanisms and Limitations

TL;DR

This paper investigates mechanisms to mitigate bias caused by subgroup underrepresentation in medical image classification, proposing a novel robust optimization approach to improve invariant representation learning and reduce performance disparities.

Contribution

It introduces a new robust optimization method for invariant representation learning to address subgroup underrepresentation bias in deep neural networks.

Findings

Reweighting underperforming samples can be problematic when bias isn't the only issue.

Naive invariant representation learning can itself be biased by spurious correlations.

The proposed robust optimization approach improves subgroup fairness while maintaining overall performance.

Abstract

Spurious correlation caused by subgroup underrepresentation has received increasing attention as a source of bias that can be perpetuated by deep neural networks (DNNs). Distributionally robust optimization has shown success in addressing this bias, although the underlying working mechanism mostly relies on upweighting under-performing samples as surrogates for those underrepresented in data. At the same time, while invariant representation learning has been a powerful choice for removing nuisance-sensitive features, it has been little considered in settings where spurious correlations are caused by significant underrepresentation of subgroups. In this paper, we take the first step to better understand and improve the mechanisms for debiasing spurious correlation due to subgroup underrepresentation in medical image classification. Through a comprehensive evaluation study, we first show that 1) generalized reweighting of under-performing samples can be problematic when bias is not the only cause for poor performance, while 2) naive invariant representation learning suffers from spurious correlations itself. We then present a novel approach that leverages robust optimization to facilitate the learning of invariant representations at the presence of spurious correlations. Finetuned classifiers utilizing such representation demonstrated improved abilities to reduce subgroup performance disparity, while maintaining high average and worst-group performance.