Counterfactual Fairness by Combining Factual and Counterfactual Predictions

TL;DR

This paper investigates the trade-off between counterfactual fairness and predictive accuracy in machine learning models, proposing methods to achieve fairness without sacrificing optimality and analyzing their performance under causal knowledge constraints.

Contribution

It introduces a theoretical framework for understanding the fairness-performance trade-off and proposes a practical algorithm to enforce counterfactual fairness with limited causal information.

Findings

Theoretical quantification of the fairness-performance trade-off.

A method to convert optimal predictors into fair ones without losing optimality.

Experimental validation on synthetic datasets supports the analysis.

Abstract

In high-stake domains such as healthcare and hiring, the role of machine learning (ML) in decision-making raises significant fairness concerns. This work focuses on Counterfactual Fairness (CF), which posits that an ML model's outcome on any individual should remain unchanged if they had belonged to a different demographic group. Previous works have proposed methods that guarantee CF. Notwithstanding, their effects on the model's predictive performance remains largely unclear. To fill in this gap, we provide a theoretical study on the inherent trade-off between CF and predictive performance in a model-agnostic manner. We first propose a simple but effective method to cast an optimal but potentially unfair predictor into a fair one without losing the optimality. By analyzing its excess risk in order to achieve CF, we quantify this inherent trade-off. Further analysis on our method's performance with access to only incomplete causal knowledge is also conducted. Built upon it, we propose a performant algorithm that can be applied in such scenarios. Experiments on both synthetic and semi-synthetic datasets demonstrate the validity of our analysis and methods.