Adaptive Data Debiasing through Bounded Exploration

TL;DR

This paper introduces an adaptive algorithm for sequentially debiasing datasets in classification tasks by balancing exploration and exploitation, aiming to improve fairness and accuracy while managing exploration risks.

Contribution

It proposes a novel bounded exploration algorithm for data debiasing that balances fairness, accuracy, and exploration risks in a sequential decision-making context.

Findings

Exploration can effectively reduce data biases in certain distributions.

The algorithm balances fairness and accuracy through adjustable parameters.

Experimental results demonstrate improved fairness and decision quality.

Abstract

Biases in existing datasets used to train algorithmic decision rules can raise ethical and economic concerns due to the resulting disparate treatment of different groups. We propose an algorithm for sequentially debiasing such datasets through adaptive and bounded exploration in a classification problem with costly and censored feedback. Exploration in this context means that at times, and to a judiciously-chosen extent, the decision maker deviates from its (current) loss-minimizing rule, and instead accepts some individuals that would otherwise be rejected, so as to reduce statistical data biases. Our proposed algorithm includes parameters that can be used to balance between the ultimate goal of removing data biases -- which will in turn lead to more accurate and fair decisions, and the exploration risks incurred to achieve this goal. We analytically show that such exploration can help debias data in certain distributions. We further investigate how fairness criteria can work in conjunction with our data debiasing algorithm. We illustrate the performance of our algorithm using experiments on synthetic and real-world datasets.