Explanatory Interactive Machine Learning for Bias Mitigation in Visual Gender Classification

TL;DR

This paper investigates how explanatory interactive learning (XIL) can be used to reduce bias and improve fairness in visual gender classification models by guiding models to focus on relevant features and mitigate spurious correlations.

Contribution

It compares two state-of-the-art XIL strategies, CAIPI and RRR, introduces a hybrid approach, and demonstrates their effectiveness in reducing bias and increasing fairness in gender classifiers.

Findings

XIL methods guide models to focus on relevant features.

CAIPI can improve classification accuracy.

XIL reduces model bias and enhances fairness.

Abstract

Explanatory interactive learning (XIL) enables users to guide model training in machine learning (ML) by providing feedback on the model's explanations, thereby helping it to focus on features that are relevant to the prediction from the user's perspective. In this study, we explore the capability of this learning paradigm to mitigate bias and spurious correlations in visual classifiers, specifically in scenarios prone to data bias, such as gender classification. We investigate two methodologically different state-of-the-art XIL strategies, i.e., CAIPI and Right for the Right Reasons (RRR), as well as a novel hybrid approach that combines both strategies. The results are evaluated quantitatively by comparing segmentation masks with explanations generated using Gradient-weighted Class Activation Mapping (GradCAM) and Bounded Logit Attention (BLA). Experimental results demonstrate the effectiveness of these methods in (i) guiding ML models to focus on relevant image features, particularly when CAIPI is used, and (ii) reducing model bias (i.e., balancing the misclassification rates between male and female predictions). Our analysis further supports the potential of XIL methods to improve fairness in gender classifiers. Overall, the increased transparency and fairness obtained by XIL leads to slight performance decreases with an exception being CAIPI, which shows potential to even improve classification accuracy.