ABCinML: Anticipatory Bias Correction in Machine Learning Applications

TL;DR

This paper introduces an anticipatory bias correction method in machine learning that predicts and adjusts for future distribution shifts to maintain fairness over time.

Contribution

It proposes a novel anticipatory dynamic learning approach that uses predictions of future subgroup distributions to preemptively mitigate bias.

Findings

Effective in reducing bias in real-world datasets

Maintains fairness better over time compared to reactive methods

Shows promise for deployment in dynamic environments

Abstract

The idealization of a static machine-learned model, trained once and deployed forever, is not practical. As input distributions change over time, the model will not only lose accuracy, any constraints to reduce bias against a protected class may fail to work as intended. Thus, researchers have begun to explore ways to maintain algorithmic fairness over time. One line of work focuses on dynamic learning: retraining after each batch, and the other on robust learning which tries to make algorithms robust against all possible future changes. Dynamic learning seeks to reduce biases soon after they have occurred and robust learning often yields (overly) conservative models. We propose an anticipatory dynamic learning approach for correcting the algorithm to mitigate bias before it occurs. Specifically, we make use of anticipations regarding the relative distributions of population subgroups (e.g., relative ratios of male and female applicants) in the next cycle to identify the right parameters for an importance weighing fairness approach. Results from experiments over multiple real-world datasets suggest that this approach has promise for anticipatory bias correction.