Measuring Unfairness through Game-Theoretic Interpretability

TL;DR

This paper explores the relationship between fairness measures and feature importance, specifically evaluating and comparing them using SHAP across datasets prone to unfairness.

Contribution

It introduces methods to evaluate and compare fairness and feature importance measures, focusing on SHAP, filling a gap in existing research.

Findings

SHAP effectively highlights unfairness in datasets.

Comparison methods reveal differences between fairness and feature importance measures.

Results suggest potential for better interpretability of unfairness in classifiers.

Abstract

One often finds in the literature connections between measures of fairness and measures of feature importance employed to interpret trained classifiers. However, there seems to be no study that compares fairness measures and feature importance measures. In this paper we propose ways to evaluate and compare such measures. We focus in particular on SHAP, a game-theoretic measure of feature importance; we present results for a number of unfairness-prone datasets.