Discover and Mitigate Unknown Biases with Debiasing Alternate Networks

TL;DR

This paper introduces DebiAN, a novel method with two networks that discover and mitigate unknown biases in image classifiers without needing bias labels, effectively handling multiple biases in complex datasets.

Contribution

DebiAN is the first approach to identify and mitigate multiple unknown biases simultaneously without bias annotations, improving fairness in deep image classifiers.

Findings

DebiAN effectively discovers unknown biases in real-world datasets.

It achieves superior bias mitigation compared to previous methods.

DebiAN handles multiple biases better than single-bias evaluation methods.

Abstract

Deep image classifiers have been found to learn biases from datasets. To mitigate the biases, most previous methods require labels of protected attributes (e.g., age, skin tone) as full-supervision, which has two limitations: 1) it is infeasible when the labels are unavailable; 2) they are incapable of mitigating unknown biases -- biases that humans do not preconceive. To resolve those problems, we propose Debiasing Alternate Networks (DebiAN), which comprises two networks -- a Discoverer and a Classifier. By training in an alternate manner, the discoverer tries to find multiple unknown biases of the classifier without any annotations of biases, and the classifier aims at unlearning the biases identified by the discoverer. While previous works evaluate debiasing results in terms of a single bias, we create Multi-Color MNIST dataset to better benchmark mitigation of multiple biases in a multi-bias setting, which not only reveals the problems in previous methods but also demonstrates the advantage of DebiAN in identifying and mitigating multiple biases simultaneously. We further conduct extensive experiments on real-world datasets, showing that the discoverer in DebiAN can identify unknown biases that may be hard to be found by humans. Regarding debiasing, DebiAN achieves strong bias mitigation performance.