Individual Fairness Through Reweighting and Tuning

TL;DR

This paper explores defining Graph Laplacian Regularizer independently on train and target data to improve individual fairness in AI, introduces a new fairness metric, and compares it with existing measures to better evaluate fairness enhancements.

Contribution

It proposes a novel approach to applying GLR separately on train and target data and introduces the Normalized Fairness Gain score for more accurate fairness measurement.

Findings

GLR applied independently on train and target data achieves similar accuracy to combined approach.

NFG score provides a more reliable measure of fairness improvement than Prediction Consistency.

The study reveals potential misleading aspects of Prediction Consistency as a fairness metric.

Abstract

Inherent bias within society can be amplified and perpetuated by artificial intelligence (AI) systems. To address this issue, a wide range of solutions have been proposed to identify and mitigate bias and enforce fairness for individuals and groups. Recently, Graph Laplacian Regularizer (GLR), a regularization technique from the semi-supervised learning literature has been used as a substitute for the common Lipschitz condition to enhance individual fairness. Notable prior work has shown that enforcing individual fairness through a GLR can improve the transfer learning accuracy of AI models under covariate shifts. However, the prior work defines a GLR on the source and target data combined, implicitly assuming that the target data are available at train time, which might not hold in practice. In this work, we investigated whether defining a GLR independently on the train and target data could maintain similar accuracy. Furthermore, we introduced the Normalized Fairness Gain score (NFG) to measure individual fairness by measuring the amount of gained fairness when a GLR is used versus not. We evaluated the new and original methods under NFG, the Prediction Consistency (PC), and traditional classification metrics on the German Credit Approval dataset. The results showed that the two models achieved similar statistical mean performances over five-fold cross-validation. Furthermore, the proposed metric showed that PC scores can be misleading as the scores can be high and statistically similar to fairness-enhanced models while NFG scores are small. This work therefore provides new insights into when a GLR effectively enhances individual fairness and the pitfalls of PC.