Post-hoc Bias Scoring Is Optimal For Fair Classification

TL;DR

This paper introduces a simple post-hoc bias scoring method for fair classification that optimally adjusts classifiers under various group fairness constraints, maintaining high accuracy.

Contribution

It provides an explicit characterization of the Bayes optimal classifier under fairness constraints and introduces a bias score for effective post-hoc adjustments.

Findings

Bias score enables optimal fairness adjustments

Thresholding bias scores achieves fairness-accuracy trade-offs

Method applies to multiple fairness criteria and sensitive attributes

Abstract

We consider a binary classification problem under group fairness constraints, which can be one of Demographic Parity (DP), Equalized Opportunity (EOp), or Equalized Odds (EO). We propose an explicit characterization of Bayes optimal classifier under the fairness constraints, which turns out to be a simple modification rule of the unconstrained classifier. Namely, we introduce a novel instance-level measure of bias, which we call bias score, and the modification rule is a simple linear rule on top of the finite amount of bias scores.Based on this characterization, we develop a post-hoc approach that allows us to adapt to fairness constraints while maintaining high accuracy. In the case of DP and EOp constraints, the modification rule is thresholding a single bias score, while in the case of EO constraints we are required to fit a linear modification rule with 2 parameters. The method can also be applied for composite group-fairness criteria, such as ones involving several sensitive attributes.