InterFair: Debiasing with Natural Language Feedback for Fair Interpretable Predictions

TL;DR

This paper proposes an interactive approach to debias NLP models using natural language feedback, enabling better bias mitigation and task performance without removing sensitive information entirely.

Contribution

It introduces two user-in-the-loop setups that leverage natural language feedback to achieve fairer and more effective debiasing in NLP models.

Findings

Bias in explanations decreased by 5-8% with maintained accuracy.

Human feedback disentangled bias from predictive info, improving bias mitigation.

Task performance improved by 4-5% through interactive feedback.

Abstract

Debiasing methods in NLP models traditionally focus on isolating information related to a sensitive attribute (e.g., gender or race). We instead argue that a favorable debiasing method should use sensitive information 'fairly,' with explanations, rather than blindly eliminating it. This fair balance is often subjective and can be challenging to achieve algorithmically. We explore two interactive setups with a frozen predictive model and show that users able to provide feedback can achieve a better and fairer balance between task performance and bias mitigation. In one setup, users, by interacting with test examples, further decreased bias in the explanations (5-8%) while maintaining the same prediction accuracy. In the other setup, human feedback was able to disentangle associated bias and predictive information from the input leading to superior bias mitigation and improved task performance (4-5%) simultaneously.