A Critical Review of Predominant Bias in Neural Networks

TL;DR

This paper clarifies two main biases in neural networks—performance fairness across demographics and non-reliance on protected attributes—by providing formal definitions, analyzing their differences, and evaluating mitigation strategies to improve fairness research.

Contribution

It introduces formal mathematical definitions for the two biases, unifies existing literature, and offers experimental validation and guidance to reduce confusion in bias mitigation research.

Findings

Distinct nature of the two biases validated through experiments

Evaluation of bias assessment metrics across datasets

Guidelines to improve clarity in bias research

Abstract

Bias issues of neural networks garner significant attention along with its promising advancement. Among various bias issues, mitigating two predominant biases is crucial in advancing fair and trustworthy AI: (1) ensuring neural networks yields even performance across demographic groups, and (2) ensuring algorithmic decision-making does not rely on protected attributes. However, upon the investigation of \pc papers in the relevant literature, we find that there exists a persistent, extensive but under-explored confusion regarding these two types of biases. Furthermore, the confusion has already significantly hampered the clarity of the community and subsequent development of debiasing methodologies. Thus, in this work, we aim to restore clarity by providing two mathematical definitions for these two predominant biases and leveraging these definitions to unify a comprehensive list of papers. Next, we highlight the common phenomena and the possible reasons for the existing confusion. To alleviate the confusion, we provide extensive experiments on synthetic, census, and image datasets, to validate the distinct nature of these biases, distinguish their different real-world manifestations, and evaluate the effectiveness of a comprehensive list of bias assessment metrics in assessing the mitigation of these biases. Further, we compare these two types of biases from multiple dimensions including the underlying causes, debiasing methods, evaluation protocol, prevalent datasets, and future directions. Last, we provide several suggestions aiming to guide researchers engaged in bias-related work to avoid confusion and further enhance clarity in the community.