Fair inference on error-prone outcomes

TL;DR

This paper addresses fairness in supervised learning when target labels are error-prone, proposing a framework combining fair ML and measurement models to correct unfairness caused by measurement error.

Contribution

It introduces a novel framework that integrates fair ML methods with measurement models to handle unfairness due to label measurement errors.

Findings

Using a latent variable model reduces unfairness caused by measurement error.

Existing fairness assessment methods do not extend well to error-prone targets.

The proposed approach improves fairness in healthcare decision-making scenarios.

Abstract

Fair inference in supervised learning is an important and active area of research, yielding a range of useful methods to assess and account for fairness criteria when predicting ground truth targets. As shown in recent work, however, when target labels are error-prone, potential prediction unfairness can arise from measurement error. In this paper, we show that, when an error-prone proxy target is used, existing methods to assess and calibrate fairness criteria do not extend to the true target variable of interest. To remedy this problem, we suggest a framework resulting from the combination of two existing literatures: fair ML methods, such as those found in the counterfactual fairness literature on the one hand, and, on the other, measurement models found in the statistical literature. We discuss these approaches and their connection resulting in our framework. In a healthcare decision problem, we find that using a latent variable model to account for measurement error removes the unfairness detected previously.