Active Fairness in Algorithmic Decision Making

TL;DR

This paper introduces an active, adaptive approach to fair classification that improves upon existing randomized methods by selectively acquiring information to balance disparities in error rates and calibration across groups and individuals.

Contribution

It proposes two novel active frameworks for fair classification that adapt information collection to group and individual needs, outperforming prior randomized methods.

Findings

Achieves calibration and single error parity (e.g., equal opportunity).

Attains parity in false positive and false negative rates (equal odds).

Outperforms randomization-based classifiers, reducing intra-group unfairness.

Abstract

Society increasingly relies on machine learning models for automated decision making. Yet, efficiency gains from automation have come paired with concern for algorithmic discrimination that can systematize inequality. Recent work has proposed optimal post-processing methods that randomize classification decisions for a fraction of individuals, in order to achieve fairness measures related to parity in errors and calibration. These methods, however, have raised concern due to the information inefficiency, intra-group unfairness, and Pareto sub-optimality they entail. The present work proposes an alternative active framework for fair classification, where, in deployment, a decision-maker adaptively acquires information according to the needs of different groups or individuals, towards balancing disparities in classification performance. We propose two such methods, where information collection is adapted to group- and individual-level needs respectively. We show on real-world datasets that these can achieve: 1) calibration and single error parity (e.g., equal opportunity); and 2) parity in both false positive and false negative rates (i.e., equal odds). Moreover, we show that by leveraging their additional degree of freedom, active approaches can substantially outperform randomization-based classifiers previously considered optimal, while avoiding limitations such as intra-group unfairness.