The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric

TL;DR

This paper extends fairness analysis of risk scores beyond binary classification to bipartite ranking, introducing the xAUC disparity metric to detect disparities in ranking tasks across protected groups.

Contribution

It proposes the xAUC disparity metric for assessing fairness in bipartite ranking and provides a decomposition of ranking loss to analyze sources of disparity.

Findings

xAUC disparity reveals biases not seen in within-group performance

Decomposition isolates sources of ranking disparity

Audits on real datasets demonstrate the metric's effectiveness

Abstract

Where machine-learned predictive risk scores inform high-stakes decisions, such as bail and sentencing in criminal justice, fairness has been a serious concern. Recent work has characterized the disparate impact that such risk scores can have when used for a binary classification task. This may not account, however, for the more diverse downstream uses of risk scores and their non-binary nature. To better account for this, in this paper, we investigate the fairness of predictive risk scores from the point of view of a bipartite ranking task, where one seeks to rank positive examples higher than negative ones. We introduce the xAUC disparity as a metric to assess the disparate impact of risk scores and define it as the difference in the probabilities of ranking a random positive example from one protected group above a negative one from another group and vice versa. We provide a decomposition of bipartite ranking loss into components that involve the discrepancy and components that involve pure predictive ability within each group. We use xAUC analysis to audit predictive risk scores for recidivism prediction, income prediction, and cardiac arrest prediction, where it describes disparities that are not evident from simply comparing within-group predictive performance.