Evaluating Debiasing Techniques for Intersectional Biases

TL;DR

This paper assesses debiasing techniques in NLP for intersectional biases, emphasizing the importance of considering multi-attribute groups to achieve fairer models, and introduces new methods for this purpose.

Contribution

It introduces a novel bias-constrained model and extends the iterative nullspace projection technique to handle multiple protected attributes in NLP.

Findings

Debiasing methods need to account for intersectional groups.

Extended nullspace projection effectively reduces intersectional bias.

New bias-constrained model improves fairness in NLP models.

Abstract

Bias is pervasive in NLP models, motivating the development of automatic debiasing techniques. Evaluation of NLP debiasing methods has largely been limited to binary attributes in isolation, e.g., debiasing with respect to binary gender or race, however many corpora involve multiple such attributes, possibly with higher cardinality. In this paper we argue that a truly fair model must consider `gerrymandering' groups which comprise not only single attributes, but also intersectional groups. We evaluate a form of bias-constrained model which is new to NLP, as well an extension of the iterative nullspace projection technique which can handle multiple protected attributes.