Trusting Fair Data: Leveraging Quality in Fairness-Driven Data Removal Techniques

TL;DR

This paper introduces a multi-objective optimization approach to fair data removal that balances fairness with data retention, enhancing trustworthiness of bias mitigation techniques in machine learning.

Contribution

It proposes a novel multi-objective framework and Pareto-optimal solutions for fair data removal, addressing trustworthiness and data quality concerns.

Findings

Balances fairness and data retention effectively

Provides Pareto-optimal solutions for data subset selection

Distributed as a Python package for practical use

Abstract

In this paper, we deal with bias mitigation techniques that remove specific data points from the training set to aim for a fair representation of the population in that set. Machine learning models are trained on these pre-processed datasets, and their predictions are expected to be fair. However, such approaches may exclude relevant data, making the attained subsets less trustworthy for further usage. To enhance the trustworthiness of prior methods, we propose additional requirements and objectives that the subsets must fulfill in addition to fairness: (1) group coverage, and (2) minimal data loss. While removing entire groups may improve the measured fairness, this practice is very problematic as failing to represent every group cannot be considered fair. In our second concern, we advocate for the retention of data while minimizing discrimination. By introducing a multi-objective optimization problem that considers fairness and data loss, we propose a methodology to find Pareto-optimal solutions that balance these objectives. By identifying such solutions, users can make informed decisions about the trade-off between fairness and data quality and select the most suitable subset for their application. Our method is distributed as a Python package via PyPI under the name FairDo (https://github.com/mkduong-ai/fairdo).