Measuring and Mitigating Bias for Tabular Datasets with Multiple Protected Attributes

TL;DR

This paper introduces new measures for assessing bias in tabular datasets with multiple protected attributes and demonstrates a mitigation strategy that reduces discrimination without significantly affecting model performance.

Contribution

It proposes new discrimination measures and applies an existing bias mitigation method, FairDo, to effectively reduce intersectional bias in real-world datasets.

Findings

Bias reduction averaged 28% across datasets

Mitigation does not significantly impact model accuracy

New measures help evaluate fairness in complex datasets

Abstract

Motivated by the recital (67) of the current corrigendum of the AI Act in the European Union, we propose and present measures and mitigation strategies for discrimination in tabular datasets. We specifically focus on datasets that contain multiple protected attributes, such as nationality, age, and sex. This makes measuring and mitigating bias more challenging, as many existing methods are designed for a single protected attribute. This paper comes with a twofold contribution: Firstly, new discrimination measures are introduced. These measures are categorized in our framework along with existing ones, guiding researchers and practitioners in choosing the right measure to assess the fairness of the underlying dataset. Secondly, a novel application of an existing bias mitigation method, FairDo, is presented. We show that this strategy can mitigate any type of discrimination, including intersectional discrimination, by transforming the dataset. By conducting experiments on real-world datasets (Adult, Bank, COMPAS), we demonstrate that de-biasing datasets with multiple protected attributes is possible. All transformed datasets show a reduction in discrimination, on average by 28%. Further, these datasets do not compromise any of the tested machine learning models' performances significantly compared to the original datasets. Conclusively, this study demonstrates the effectiveness of the mitigation strategy used and contributes to the ongoing discussion on the implementation of the European Union's AI Act.