Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey

TL;DR

This comprehensive survey reviews 341 publications on bias mitigation techniques in machine learning classifiers, analyzing their methods, evaluation practices, and datasets to guide practitioners in developing fair models.

Contribution

It systematically categorizes bias mitigation methods, evaluates their evaluation practices, and provides insights to inform future research and practical applications.

Findings

Pre-processing, in-processing, and post-processing are key intervention procedures.

Familiarity with datasets and metrics is crucial for evaluating bias mitigation methods.

The survey identifies the most popular fairness metrics and datasets used in the field.

Abstract

This paper provides a comprehensive survey of bias mitigation methods for achieving fairness in Machine Learning (ML) models. We collect a total of 341 publications concerning bias mitigation for ML classifiers. These methods can be distinguished based on their intervention procedure (i.e., pre-processing, in-processing, post-processing) and the technique they apply. We investigate how existing bias mitigation methods are evaluated in the literature. In particular, we consider datasets, metrics and benchmarking. Based on the gathered insights (e.g., What is the most popular fairness metric? How many datasets are used for evaluating bias mitigation methods?), we hope to support practitioners in making informed choices when developing and evaluating new bias mitigation methods.