On Fairness and Calibration

TL;DR

This paper explores the inherent conflict between fairness and calibration in classification models, revealing that achieving both simultaneously is highly restrictive and often leads to trivial solutions, with empirical validation on multiple datasets.

Contribution

It demonstrates that calibration can only be compatible with a single fairness constraint and that satisfying this leads to trivial or randomized classifiers, extending prior theoretical results.

Findings

Calibration is only compatible with equal false-negative rates across groups.

Achieving fairness and calibration simultaneously often results in trivial classifiers.

Empirical results confirm the theoretical limitations on fairness and calibration trade-offs.

Abstract

The machine learning community has become increasingly concerned with the potential for bias and discrimination in predictive models. This has motivated a growing line of work on what it means for a classification procedure to be "fair." In this paper, we investigate the tension between minimizing error disparity across different population groups while maintaining calibrated probability estimates. We show that calibration is compatible only with a single error constraint (i.e. equal false-negatives rates across groups), and show that any algorithm that satisfies this relaxation is no better than randomizing a percentage of predictions for an existing classifier. These unsettling findings, which extend and generalize existing results, are empirically confirmed on several datasets.