Group Robust Classification Without Any Group Information

TL;DR

This paper introduces a bias-unsupervised method for robust classification that leverages pretrained self-supervised models to improve accuracy without relying on group labels, addressing limitations of existing approaches.

Contribution

The study proposes a novel bias-unsupervised training and validation approach using pretrained models, eliminating the need for bias annotations and improving robustness.

Findings

Outperforms existing bias-unsupervised methods on synthetic and real-world datasets.

Effectively generalizes to unseen attribute combinations in the MPI3D dataset.

Achieves competitive or superior robust accuracy compared to state-of-the-art methods.

Abstract

Empirical risk minimization (ERM) is sensitive to spurious correlations in the training data, which poses a significant risk when deploying systems trained under this paradigm in high-stake applications. While the existing literature focuses on maximizing group-balanced or worst-group accuracy, estimating these accuracies is hindered by costly bias annotations. This study contends that current bias-unsupervised approaches to group robustness continue to rely on group information to achieve optimal performance. Firstly, these methods implicitly assume that all group combinations are represented during training. To illustrate this, we introduce a systematic generalization task on the MPI3D dataset and discover that current algorithms fail to improve the ERM baseline when combinations of observed attribute values are missing. Secondly, bias labels are still crucial for effective model selection, restricting the practicality of these methods in real-world scenarios. To address these limitations, we propose a revised methodology for training and validating debiased models in an entirely bias-unsupervised manner. We achieve this by employing pretrained self-supervised models to reliably extract bias information, which enables the integration of a logit adjustment training loss with our validation criterion. Our empirical analysis on synthetic and real-world tasks provides evidence that our approach overcomes the identified challenges and consistently enhances robust accuracy, attaining performance which is competitive with or outperforms that of state-of-the-art methods, which, conversely, rely on bias labels for validation.