Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features

TL;DR

This paper introduces Shortcut Debiasing, a novel method that uses controllable shortcut features to replace bias features in training, enabling effective bias removal during inference and improving fairness in visual recognition.

Contribution

The paper proposes a new debiasing approach leveraging shortcut features to transfer bias learning and facilitate causal intervention, addressing limitations of existing methods.

Findings

Achieves significant fairness improvements over state-of-the-art methods.

Maintains high accuracy while removing bias features.

Effective across multiple benchmark datasets.

Abstract

Machine learning models often learn to make predictions that rely on sensitive social attributes like gender and race, which poses significant fairness risks, especially in societal applications, such as hiring, banking, and criminal justice. Existing work tackles this issue by minimizing the employed information about social attributes in models for debiasing. However, the high correlation between target task and these social attributes makes learning on the target task incompatible with debiasing. Given that model bias arises due to the learning of bias features (\emph{i.e}., gender) that help target task optimization, we explore the following research question: \emph{Can we leverage shortcut features to replace the role of bias feature in target task optimization for debiasing?} To this end, we propose \emph{Shortcut Debiasing}, to first transfer the target task's learning of bias attributes from bias features to shortcut features, and then employ causal intervention to eliminate shortcut features during inference. The key idea of \emph{Shortcut Debiasing} is to design controllable shortcut features to on one hand replace bias features in contributing to the target task during the training stage, and on the other hand be easily removed by intervention during the inference stage. This guarantees the learning of the target task does not hinder the elimination of bias features. We apply \emph{Shortcut Debiasing} to several benchmark datasets, and achieve significant improvements over the state-of-the-art debiasing methods in both accuracy and fairness.