Subgroups Matter for Robust Bias Mitigation

TL;DR

This paper investigates how the choice of subgroups in bias mitigation methods critically affects their success, revealing that improper subgroup definitions can worsen fairness and proposing that alternative subgroup choices can enhance robustness.

Contribution

It systematically evaluates the impact of subgroup definitions on bias mitigation effectiveness across vision and language tasks, providing theoretical insights and practical guidelines.

Findings

Subgroup choice significantly influences bias mitigation outcomes.

Certain subgroupings can lead to worse results than no mitigation.

Using different subgroup sets can improve fairness in some cases.

Abstract

Despite the constant development of new bias mitigation methods for machine learning, no method consistently succeeds, and a fundamental question remains unanswered: when and why do bias mitigation techniques fail? In this paper, we hypothesise that a key factor may be the often-overlooked but crucial step shared by many bias mitigation methods: the definition of subgroups. To investigate this, we conduct a comprehensive evaluation of state-of-the-art bias mitigation methods across multiple vision and language classification tasks, systematically varying subgroup definitions, including coarse, fine-grained, intersectional, and noisy subgroups. Our results reveal that subgroup choice significantly impacts performance, with certain groupings paradoxically leading to worse outcomes than no mitigation at all. Our findings suggest that observing a disparity between a set of subgroups is not a sufficient reason to use those subgroups for mitigation. Through theoretical analysis, we explain these phenomena and uncover a counter-intuitive insight that, in some cases, improving fairness with respect to a particular set of subgroups is best achieved by using a different set of subgroups for mitigation. Our work highlights the importance of careful subgroup definition in bias mitigation and presents it as an alternative lever for improving the robustness and fairness of machine learning models.