Directional Bias Amplification

TL;DR

This paper introduces a new metric called Directional Bias Amplification to better measure how machine learning models amplify biases from data, addressing shortcomings of previous metrics and discussing its implications.

Contribution

The paper proposes a decoupled, more accurate metric for bias amplification and analyzes its assumptions, limitations, and normative implications.

Findings

The new metric better isolates bias amplification effects.

It highlights the importance of confidence intervals in measurement.

The analysis discusses the normative considerations of bias measurement.

Abstract

Mitigating bias in machine learning systems requires refining our understanding of bias propagation pathways: from societal structures to large-scale data to trained models to impact on society. In this work, we focus on one aspect of the problem, namely bias amplification: the tendency of models to amplify the biases present in the data they are trained on. A metric for measuring bias amplification was introduced in the seminal work by Zhao et al. (2017); however, as we demonstrate, this metric suffers from a number of shortcomings including conflating different types of bias amplification and failing to account for varying base rates of protected attributes. We introduce and analyze a new, decoupled metric for measuring bias amplification, $\text{BiasAmp}_{\rightarrow}$ (Directional Bias Amplification). We thoroughly analyze and discuss both the technical assumptions and normative implications of this metric. We provide suggestions about its measurement by cautioning against predicting sensitive attributes, encouraging the use of confidence intervals due to fluctuations in the fairness of models across runs, and discussing the limitations of what this metric captures. Throughout this paper, we work to provide an interrogative look at the technical measurement of bias amplification, guided by our normative ideas of what we want it to encompass. Code is located at https://github.com/princetonvisualai/directional-bias-amp