Transparency and Proportionality in Post-Processing Algorithmic Bias Correction

TL;DR

This paper introduces measures to evaluate the fairness and transparency of post-processing bias correction methods in algorithmic decision-making, aiming to improve fairness assessments and guide better bias mitigation strategies.

Contribution

It develops new metrics for quantifying the proportionality and transparency of post-processing fairness interventions, enhancing understanding of their impacts.

Findings

Metrics effectively assess fairness and transparency in post-processing bias correction

Applying these measures reveals insights beyond traditional fairness metrics

The methodology supports more informed bias mitigation decisions

Abstract

Algorithmic decision-making systems sometimes produce errors or skewed predictions toward a particular group, leading to unfair results. Debiasing practices, applied at different stages of the development of such systems, occasionally introduce new forms of unfairness or exacerbate existing inequalities. We focus on post-processing techniques that modify algorithmic predictions to achieve fairness in classification tasks, examining the unintended consequences of these interventions. To address this challenge, we develop a set of measures that quantify the disparity in the flips applied to the solution in the post-processing stage. The proposed measures will help practitioners: (1) assess the proportionality of the debiasing strategy used, (2) have transparency to explain the effects of the strategy in each group, and (3) based on those results, analyze the possibility of the use of some other approaches for bias mitigation or to solve the problem. We introduce a methodology for applying the proposed metrics during the post-processing stage and illustrate its practical application through an example. This example demonstrates how analyzing the proportionality of the debiasing strategy complements traditional fairness metrics, providing a deeper perspective to ensure fairer outcomes across all groups.