Achieving Equalized Odds by Resampling Sensitive Attributes

TL;DR

This paper introduces a flexible, differentiable framework for training predictive models that approximately satisfy equalized odds fairness, including a novel hypothesis test and methods for equitable uncertainty quantification.

Contribution

It proposes a new discrepancy functional for fairness, a formal hypothesis test for violations, and techniques for equitable uncertainty quantification, advancing fairness in regression and classification.

Findings

Improved fairness performance over state-of-the-art methods

First formal hypothesis test for equalized odds violations

Framework applicable to regression and multi-class classification

Abstract

We present a flexible framework for learning predictive models that approximately satisfy the equalized odds notion of fairness. This is achieved by introducing a general discrepancy functional that rigorously quantifies violations of this criterion. This differentiable functional is used as a penalty driving the model parameters towards equalized odds. To rigorously evaluate fitted models, we develop a formal hypothesis test to detect whether a prediction rule violates this property, the first such test in the literature. Both the model fitting and hypothesis testing leverage a resampled version of the sensitive attribute obeying equalized odds, by construction. We demonstrate the applicability and validity of the proposed framework both in regression and multi-class classification problems, reporting improved performance over state-of-the-art methods. Lastly, we show how to incorporate techniques for equitable uncertainty quantification---unbiased for each group under study---to communicate the results of the data analysis in exact terms.