BiMi Sheets: Infosheets for bias mitigation methods

TL;DR

BiMi Sheets provides a standardized, portable documentation framework for bias mitigation methods in machine learning, facilitating comparison, understanding, and adoption across diverse domains and tasks.

Contribution

The paper introduces BiMi Sheets, a structured guide for documenting bias mitigation methods, addressing benchmarking challenges and promoting wider adoption.

Findings

Enables quick comparison of bias mitigation methods.

Facilitates creation of a structured database of methods.

Supports practitioners in selecting appropriate bias mitigation strategies.

Abstract

Over the past 15 years, hundreds of bias mitigation methods have been proposed in the pursuit of fairness in machine learning (ML). However, algorithmic biases are domain-, task-, and model-specific, leading to a `portability trap': bias mitigation solutions in one context may not be appropriate in another. Thus, a myriad of design choices have to be made when creating a bias mitigation method, such as the formalization of fairness it pursues, and where and how it intervenes in the ML pipeline. This creates challenges in benchmarking and comparing the relative merits of different bias mitigation methods, and limits their uptake by practitioners. We propose BiMi Sheets as a portable, uniform guide to document the design choices of any bias mitigation method. This enables researchers and practitioners to quickly learn its main characteristics and to compare with their desiderata. Furthermore, the sheets' structure allow for the creation of a structured database of bias mitigation methods. In order to foster the sheets' adoption, we provide a platform for finding and creating BiMi Sheets at bimisheet.com.