Gender Bias in Explainability: Investigating Performance Disparity in Post-hoc Methods

TL;DR

This study reveals significant gender disparities in the performance of widely used post-hoc explanation methods across multiple NLP tasks and models, emphasizing the need for fairness considerations in explainability.

Contribution

It demonstrates that explanation disparities exist independently of training data bias, highlighting a critical overlooked aspect of fairness in model interpretability.

Findings

Gender disparities in faithfulness, robustness, and complexity of explanations.

Disparities persist even with unbiased datasets.

Implications for fairness in high-stakes AI applications.

Abstract

While research on applications and evaluations of explanation methods continues to expand, fairness of the explanation methods concerning disparities in their performance across subgroups remains an often overlooked aspect. In this paper, we address this gap by showing that, across three tasks and five language models, widely used post-hoc feature attribution methods exhibit significant gender disparity with respect to their faithfulness, robustness, and complexity. These disparities persist even when the models are pre-trained or fine-tuned on particularly unbiased datasets, indicating that the disparities we observe are not merely consequences of biased training data. Our results highlight the importance of addressing disparities in explanations when developing and applying explainability methods, as these can lead to biased outcomes against certain subgroups, with particularly critical implications in high-stakes contexts. Furthermore, our findings underscore the importance of incorporating the fairness of explanations, alongside overall model fairness and explainability, as a requirement in regulatory frameworks.