Optimized Score Transformation for Consistent Fair Classification

TL;DR

This paper introduces a method for transforming classifier scores to satisfy fairness constraints while maintaining predictive accuracy, applicable in both post-processing and pre-processing settings, with theoretical guarantees and empirical advantages.

Contribution

It presents a closed-form solution and convex optimization framework for fair score transformation, along with a practical algorithm and theoretical guarantees, advancing fair probabilistic classification methods.

Findings

Outperforms existing methods on Brier score and AUC metrics

Provides theoretical guarantees for finite samples and transformation parameters

Achieves fairness constraints with minimal impact on predictive performance

Abstract

This paper considers fair probabilistic binary classification where the outputs of primary interest are predicted probabilities, commonly referred to as scores. We formulate the problem of transforming scores to satisfy fairness constraints that are linear in conditional means of scores while minimizing a cross-entropy objective. The formulation can be applied directly to post-process classifier outputs and we also explore a pre-processing extension, thus allowing maximum freedom in selecting a classification algorithm. We derive a closed-form expression for the optimal transformed scores and a convex optimization problem for the transformation parameters. In the population limit, the transformed score function is the fairness-constrained minimizer of cross-entropy with respect to the true conditional probability of the outcome. In the finite sample setting, we propose a method called FairScoreTransformer to approach this solution using a combination of standard probabilistic classifiers and ADMM. We provide several consistency and finite-sample guarantees for FairScoreTransformer, relating to the transformation parameters and transformed score function that it obtains. Comprehensive experiments comparing to 10 existing methods show that FairScoreTransformer has advantages for score-based metrics such as Brier score and AUC while remaining competitive for binary label-based metrics such as accuracy.