Pushing the Accuracy-Group Robustness Frontier with Introspective Self-play

TL;DR

This paper introduces Introspective Self-play (ISP), a method that enhances uncertainty estimation in deep neural networks by predicting dataset bias, thereby improving active learning's ability to sample underrepresented groups and enhance fairness.

Contribution

ISP is a novel auxiliary task that improves bias-awareness and uncertainty estimates in DNNs, leading to better active learning performance on imbalanced datasets.

Findings

ISP improves tail-group sampling rate.

ISP enhances accuracy-fairness trade-off.

ISP consistently benefits active learning methods.

Abstract

Standard empirical risk minimization (ERM) training can produce deep neural network (DNN) models that are accurate on average but under-perform in under-represented population subgroups, especially when there are imbalanced group distributions in the long-tailed training data. Therefore, approaches that improve the accuracy-group robustness trade-off frontier of a DNN model (i.e. improving worst-group accuracy without sacrificing average accuracy, or vice versa) is of crucial importance. Uncertainty-based active learning (AL) can potentially improve the frontier by preferentially sampling underrepresented subgroups to create a more balanced training dataset. However, the quality of uncertainty estimates from modern DNNs tend to degrade in the presence of spurious correlations and dataset bias, compromising the effectiveness of AL for sampling tail groups. In this work, we propose Introspective Self-play (ISP), a simple approach to improve the uncertainty estimation of a deep neural network under dataset bias, by adding an auxiliary introspection task requiring a model to predict the bias for each data point in addition to the label. We show that ISP provably improves the bias-awareness of the model representation and the resulting uncertainty estimates. On two real-world tabular and language tasks, ISP serves as a simple "plug-in" for AL model training, consistently improving both the tail-group sampling rate and the final accuracy-fairness trade-off frontier of popular AL methods.