Equity-Directed Bootstrapping: Examples and Analysis

TL;DR

This paper introduces an equity-directed bootstrapping method to address severe class and group imbalance in binary classification, improving fairness metrics like equal odds and odds ratios.

Contribution

It proposes a novel bootstrap approach that balances training data across classes and groups, enhancing fairness in imbalanced classification problems.

Findings

Improves test sensitivities and specificities towards equal odds.

Brings odds ratios close to one, indicating fairness.

Links to existing fairness adjustment methods.

Abstract

When faced with severely imbalanced binary classification problems, we often train models on bootstrapped data in which the number of instances of each class occur in a more favorable ratio, e.g., one. We view algorithmic inequity through the lens of imbalanced classification: in order to balance the performance of a classifier across groups, we can bootstrap to achieve training sets that are balanced with respect to both labels and group identity. For an example problem with severe class imbalance---prediction of suicide death from administrative patient records---we illustrate how an equity-directed bootstrap can bring test set sensitivities and specificities much closer to satisfying the equal odds criterion. In the context of na\"ive Bayes and logistic regression, we analyze the equity-directed bootstrap, demonstrating that it works by bringing odds ratios close to one, and linking it to methods involving intercept adjustment, thresholding, and weighting.