How Biased are Your Features?: Computing Fairness Influence Functions with Global Sensitivity Analysis

TL;DR

This paper introduces the Fairness Influence Function (FIF), a novel method to quantify how individual and intersecting features influence bias in classifiers, using global sensitivity analysis to improve fairness assessment.

Contribution

We propose FIF, a new approach that decomposes classifier bias into feature-specific components, enabling detailed fairness analysis beyond existing metrics.

Findings

FIF accurately captures feature and intersectional bias influences.

FairXplainer provides better bias approximation and correlates with fairness interventions.

The method detects bias changes due to fairness actions.

Abstract

Fairness in machine learning has attained significant focus due to the widespread application in high-stake decision-making tasks. Unregulated machine learning classifiers can exhibit bias towards certain demographic groups in data, thus the quantification and mitigation of classifier bias is a central concern in fairness in machine learning. In this paper, we aim to quantify the influence of different features in a dataset on the bias of a classifier. To do this, we introduce the Fairness Influence Function (FIF). This function breaks down bias into its components among individual features and the intersection of multiple features. The key idea is to represent existing group fairness metrics as the difference of the scaled conditional variances in the classifier's prediction and apply a decomposition of variance according to global sensitivity analysis. To estimate FIFs, we instantiate an algorithm FairXplainer that applies variance decomposition of classifier's prediction following local regression. Experiments demonstrate that FairXplainer captures FIFs of individual feature and intersectional features, provides a better approximation of bias based on FIFs, demonstrates higher correlation of FIFs with fairness interventions, and detects changes in bias due to fairness affirmative/punitive actions in the classifier. The code is available at https://github.com/ReAILe/bias-explainer.