Testing the Fairness-Accuracy Improvability of Algorithms

TL;DR

This paper introduces an econometric testing framework to assess whether algorithms can be improved in fairness without sacrificing accuracy or other objectives, addressing legal and ethical concerns.

Contribution

It provides a simple, robust test for fairness-improvability that applies broadly and is validated both theoretically and through real-world healthcare data.

Findings

The test is valid and consistent in large samples.

It can detect if fairness improvements are possible without accuracy loss.

Applied to a healthcare algorithm, it quantifies potential fairness gains.

Abstract

Many organizations use algorithms that have a disparate impact, i.e., the benefits or harms of the algorithm fall disproportionately on certain social groups. Addressing an algorithm's disparate impact can be challenging, however, because it is often unclear whether it is possible to reduce this impact without sacrificing other objectives of the organization, such as accuracy or profit. Establishing the improvability of algorithms with respect to multiple criteria is of both conceptual and practical interest: in many settings, disparate impact that would otherwise be prohibited under US federal law is permissible if it is necessary to achieve a legitimate business interest. The question is how a policy-maker can formally substantiate, or refute, this "necessity" defense. In this paper, we provide an econometric framework for testing the hypothesis that it is possible to improve on the fairness of an algorithm without compromising on other pre-specified objectives. Our proposed test is simple to implement and can be applied under any exogenous constraint on the algorithm space. We establish the large-sample validity and consistency of our test, and microfound the test's robustness to manipulation based on a game between a policymaker and the analyst. Finally, we apply our approach to evaluate a healthcare algorithm originally considered by Obermeyer et al. (2019), and quantify the extent to which the algorithm's disparate impact can be reduced without compromising the accuracy of its predictions.