Locating disparities in machine learning

TL;DR

This paper introduces ALD, a versatile data-driven framework that automatically detects disparities in machine learning outcomes across various subgroups, even with high-dimensional data and unknown sensitive attributes.

Contribution

ALD is a novel, flexible method capable of locating disparities in machine learning models regardless of classifier type, disparity definition, or predictor complexity.

Findings

ALD effectively detects disparities in synthetic datasets.

ALD successfully identifies disparities in real-world datasets.

ALD produces interpretable audit reports for practitioners.

Abstract

Machine learning can provide predictions with disparate outcomes, in which subgroups of the population (e.g., defined by age, gender, or other sensitive attributes) are systematically disadvantaged. In order to comply with upcoming legislation, practitioners need to locate such disparate outcomes. However, previous literature typically detects disparities through statistical procedures for when the sensitive attribute is specified a priori. This limits applicability in real-world settings where datasets are high dimensional and, on top of that, sensitive attributes may be unknown. As a remedy, we propose a data-driven framework called Automatic Location of Disparities (ALD) which aims at locating disparities in machine learning. ALD meets several demands from industry: ALD (1) is applicable to arbitrary machine learning classifiers; (2) operates on different definitions of disparities (e.g., statistical parity or equalized odds); and (3) deals with both categorical and continuous predictors even if disparities arise from complex and multi-way interactions known as intersectionality (e. g., age above 60 and female). ALD produces interpretable audit reports as output. We demonstrate the effectiveness of ALD based on both synthetic and real-world datasets. As a result, we empower practitioners to effectively locate and mitigate disparities in machine learning algorithms, conduct algorithmic audits, and protect individuals from discrimination.