Mitigating Spurious Correlations via Disagreement Probability

TL;DR

This paper introduces DPR, a novel bias-agnostic resampling method that improves model robustness against spurious correlations without needing bias labels, validated through empirical and theoretical analysis.

Contribution

The paper proposes a new training objective and the DPR method that effectively mitigates spurious correlations without bias labels, advancing bias mitigation techniques.

Findings

DPR achieves state-of-the-art results on multiple benchmarks.

DPR effectively identifies bias-conflicting samples without bias labels.

Theoretical analysis confirms DPR reduces dependency on spurious correlations.

Abstract

Models trained with empirical risk minimization (ERM) are prone to be biased towards spurious correlations between target labels and bias attributes, which leads to poor performance on data groups lacking spurious correlations. It is particularly challenging to address this problem when access to bias labels is not permitted. To mitigate the effect of spurious correlations without bias labels, we first introduce a novel training objective designed to robustly enhance model performance across all data samples, irrespective of the presence of spurious correlations. From this objective, we then derive a debiasing method, Disagreement Probability based Resampling for debiasing (DPR), which does not require bias labels. DPR leverages the disagreement between the target label and the prediction of a biased model to identify bias-conflicting samples-those without spurious correlations-and upsamples them according to the disagreement probability. Empirical evaluations on multiple benchmarks demonstrate that DPR achieves state-of-the-art performance over existing baselines that do not use bias labels. Furthermore, we provide a theoretical analysis that details how DPR reduces dependency on spurious correlations.