Debiasing classifiers: is reality at variance with expectation?

TL;DR

This paper empirically investigates debiasing classifiers, revealing that they often fail to generalize and can worsen fairness, due to bias-variance trade-offs and the influence of base rates, emphasizing the need for extensive validation.

Contribution

It provides a rigorous empirical analysis of debiasing methods, highlighting their limitations and explaining failures through bias-variance trade-offs and base rate effects.

Findings

Debiasing methods often do not generalize out-of-sample.

Imposing fairness constraints increases variance, affecting performance.

Partial debiasing can improve out-of-sample results due to bias-variance trade-offs.

Abstract

We present an empirical study of debiasing methods for classifiers, showing that debiasers often fail in practice to generalize out-of-sample, and can in fact make fairness worse rather than better. A rigorous evaluation of the debiasing treatment effect requires extensive cross-validation beyond what is usually done. We demonstrate that this phenomenon can be explained as a consequence of bias-variance trade-off, with an increase in variance necessitated by imposing a fairness constraint. Follow-up experiments validate the theoretical prediction that the estimation variance depends strongly on the base rates of the protected class. Considering fairness--performance trade-offs justifies the counterintuitive notion that partial debiasing can actually yield better results in practice on out-of-sample data.