Facets of Disparate Impact: Evaluating Legally Consistent Bias in Machine Learning

TL;DR

This paper introduces the Objective Fairness Index, a new metric for evaluating bias in machine learning that aligns with legal standards and helps distinguish between discrimination and systemic disparities.

Contribution

The paper proposes a novel fairness metric, the Objective Fairness Index, combining legal context and objective testing to assess bias more reliably.

Findings

The Objective Fairness Index effectively differentiates discriminatory tests from systemic disparities.

Application to COMPAS demonstrates the metric's practical utility.

The metric provides both practical and theoretical insights into bias evaluation.

Abstract

Leveraging current legal standards, we define bias through the lens of marginal benefits and objective testing with the novel metric "Objective Fairness Index". This index combines the contextual nuances of objective testing with metric stability, providing a legally consistent and reliable measure. Utilizing the Objective Fairness Index, we provide fresh insights into sensitive machine learning applications, such as COMPAS (recidivism prediction), highlighting the metric's practical and theoretical significance. The Objective Fairness Index allows one to differentiate between discriminatory tests and systemic disparities.