Fair Infinitesimal Jackknife: Mitigating the Influence of Biased Training Data Points Without Refitting

TL;DR

This paper introduces a method to enhance fairness in pre-trained classifiers by selectively removing training data points based on influence on fairness metrics, without needing to refit the model.

Contribution

It proposes a novel infinitesimal jackknife-based influence measure for data removal to improve fairness without sacrificing predictive accuracy.

Findings

Significant fairness improvements across multiple tasks

Minimal impact on predictive performance

Outperforms existing fairness intervention methods

Abstract

In consequential decision-making applications, mitigating unwanted biases in machine learning models that yield systematic disadvantage to members of groups delineated by sensitive attributes such as race and gender is one key intervention to strive for equity. Focusing on demographic parity and equality of opportunity, in this paper we propose an algorithm that improves the fairness of a pre-trained classifier by simply dropping carefully selected training data points. We select instances based on their influence on the fairness metric of interest, computed using an infinitesimal jackknife-based approach. The dropping of training points is done in principle, but in practice does not require the model to be refit. Crucially, we find that such an intervention does not substantially reduce the predictive performance of the model but drastically improves the fairness metric. Through careful experiments, we evaluate the effectiveness of the proposed approach on diverse tasks and find that it consistently improves upon existing alternatives.