A Bandit-Based Algorithm for Fairness-Aware Hyperparameter Optimization

TL;DR

Fairband is a resource-efficient, easy-to-implement bandit-based hyperparameter optimization algorithm that effectively balances fairness and accuracy in machine learning models, facilitating practical fairness-aware ML pipelines.

Contribution

We introduce Fairband, a novel bandit-based hyperparameter optimization method that incorporates fairness considerations, improving fairness outcomes with minimal accuracy loss.

Findings

Fairband efficiently finds fairness-optimized hyperparameters.

It achieves better fairness with small accuracy trade-offs.

It is simple, model-agnostic, and easy to implement.

Abstract

Considerable research effort has been guided towards algorithmic fairness but there is still no major breakthrough. In practice, an exhaustive search over all possible techniques and hyperparameters is needed to find optimal fairness-accuracy trade-offs. Hence, coupled with the lack of tools for ML practitioners, real-world adoption of bias reduction methods is still scarce. To address this, we present Fairband, a bandit-based fairness-aware hyperparameter optimization (HO) algorithm. Fairband is conceptually simple, resource-efficient, easy to implement, and agnostic to both the objective metrics, model types and the hyperparameter space being explored. Moreover, by introducing fairness notions into HO, we enable seamless and efficient integration of fairness objectives into real-world ML pipelines. We compare Fairband with popular HO methods on four real-world decision-making datasets. We show that Fairband can efficiently navigate the fairness-accuracy trade-off through hyperparameter optimization. Furthermore, without extra training cost, it consistently finds configurations attaining substantially improved fairness at a comparatively small decrease in predictive accuracy.