Bias Detection via Maximum Subgroup Discrepancy

TL;DR

This paper introduces the Maximum Subgroup Discrepancy (MSD), a new bias detection metric for AI data and outputs that is computationally feasible, interpretable, and effective in identifying biases across feature subgroups.

Contribution

The paper proposes MSD, a novel subgroup-based distance metric with linear sample complexity and a practical MIO-based evaluation algorithm, enhancing bias detection in AI systems.

Findings

MSD effectively detects biases in real-world datasets.

MSD has linear sample complexity relative to features.

MSD aligns well with a natural bias detection framework.

Abstract

Bias evaluation is fundamental to trustworthy AI, both in terms of checking data quality and in terms of checking the outputs of AI systems. In testing data quality, for example, one may study the distance of a given dataset, viewed as a distribution, to a given ground-truth reference dataset. However, classical metrics, such as the Total Variation and the Wasserstein distances, are known to have high sample complexities and, therefore, may fail to provide a meaningful distinction in many practical scenarios. In this paper, we propose a new notion of distance, the Maximum Subgroup Discrepancy (MSD). In this metric, two distributions are close if, roughly, discrepancies are low for all feature subgroups. While the number of subgroups may be exponential, we show that the sample complexity is linear in the number of features, thus making it feasible for practical applications. Moreover, we provide a practical algorithm for evaluating the distance based on Mixed-integer optimization (MIO). We also note that the proposed distance is easily interpretable, thus providing clearer paths to fixing the biases once they have been identified. Finally, we describe a natural general bias detection framework, termed MSDD distances, and show that MSD aligns well with this framework. We empirically evaluate MSD by comparing it with other metrics and by demonstrating the above properties of MSD on real-world datasets.