Fairness Evaluation in Presence of Biased Noisy Labels

TL;DR

This paper introduces a sensitivity analysis framework to evaluate how biased and noisy labels impact the fairness and bias assessment of risk prediction models in criminal justice, highlighting the importance of accounting for label noise.

Contribution

It proposes a novel framework for assessing fairness under label noise and demonstrates its application on real criminal justice datasets.

Findings

Small label biases can significantly affect fairness conclusions.

Noisy labels may lead to misleading assessments of model bias.

Sensitivity analysis reveals robustness or vulnerability of fairness metrics.

Abstract

Risk assessment tools are widely used around the country to inform decision making within the criminal justice system. Recently, considerable attention has been devoted to the question of whether such tools may suffer from racial bias. In this type of assessment, a fundamental issue is that the training and evaluation of the model is based on a variable (arrest) that may represent a noisy version of an unobserved outcome of more central interest (offense). We propose a sensitivity analysis framework for assessing how assumptions on the noise across groups affect the predictive bias properties of the risk assessment model as a predictor of reoffense. Our experimental results on two real world criminal justice data sets demonstrate how even small biases in the observed labels may call into question the conclusions of an analysis based on the noisy outcome.