Efficient Bias Mitigation Without Privileged Information

TL;DR

This paper introduces TAB, a hyperparameter-free bias mitigation method that uses a helper model's training history to identify and balance spurious samples, enhancing worst-group performance without needing group labels.

Contribution

The paper presents TAB, a novel bias mitigation framework that does not require group labels or hyperparameter tuning, leveraging training history to improve fairness.

Findings

TAB outperforms existing bias mitigation methods in worst-group accuracy.

TAB maintains overall accuracy while reducing bias.

The method requires no group annotations or extensive hyperparameter search.

Abstract

Deep neural networks trained via empirical risk minimisation often exhibit significant performance disparities across groups, particularly when group and task labels are spuriously correlated (e.g., "grassy background" and "cows"). Existing bias mitigation methods that aim to address this issue often either rely on group labels for training or validation, or require an extensive hyperparameter search. Such data and computational requirements hinder the practical deployment of these methods, especially when datasets are too large to be group-annotated, computational resources are limited, and models are trained through already complex pipelines. In this paper, we propose Targeted Augmentations for Bias Mitigation (TAB), a simple hyperparameter-free framework that leverages the entire training history of a helper model to identify spurious samples, and generate a group-balanced training set from which a robust model can be trained. We show that TAB improves worst-group performance without any group information or model selection, outperforming existing methods while maintaining overall accuracy.