Fighting Fire with Fire: Contrastive Debiasing without Bias-free Data via Generative Bias-transformation

TL;DR

This paper introduces CDvG, a contrastive debiasing method that uses generative bias transformation to learn bias-invariant representations without needing explicit bias labels or bias-free data, improving robustness in neural networks.

Contribution

The paper presents a novel bias debiasing approach that leverages image translation and contrastive learning without requiring bias annotations or bias-free samples.

Findings

Outperforms prior debiasing methods in various tasks.

Effective even with scarce or no bias-free samples.

Can be combined with existing bias-free sample methods for further gains.

Abstract

Deep neural networks (DNNs), despite their impressive ability to generalize over-capacity networks, often rely heavily on malignant bias as shortcuts instead of task-related information for discriminative tasks. To address this problem, recent studies utilize auxiliary information related to the bias, which is rarely obtainable in practice, or sift through a handful of bias-free samples for debiasing. However, the success of these methods is not always guaranteed due to the unfulfilled presumptions. In this paper, we propose a novel method, Contrastive Debiasing via Generative Bias-transformation (CDvG), which works without explicit bias labels or bias-free samples. Motivated by our observation that not only discriminative models but also image translation models tend to focus on the malignant bias, CDvG employs an image translation model to transform one bias mode into another while preserving the task-relevant information. Additionally, the bias-transformed views are set against each other through contrastive learning to learn bias-invariant representations. Our method demonstrates superior performance compared to prior approaches, especially when bias-free samples are scarce or absent. Furthermore, CDvG can be integrated with the methods that focus on bias-free samples in a plug-and-play manner for additional enhancements, as demonstrated by diverse experimental results.