Fair and Green Hyperparameter Optimization via Multi-objective and Multiple Information Source Bayesian Optimization

TL;DR

FanG-HPO introduces a multi-objective, multi-source Bayesian optimization method that efficiently balances fairness and accuracy in machine learning models while reducing energy consumption on large datasets.

Contribution

It proposes a novel approach combining multi-objective and multiple information source Bayesian optimization for fair and green hyperparameter tuning.

Findings

FanG-HPO achieves Pareto-efficient trade-offs between fairness and accuracy.

It reduces energy consumption compared to single-source methods.

Outperforms fairness-aware algorithms in experiments.

Abstract

There is a consensus that focusing only on accuracy in searching for optimal machine learning models amplifies biases contained in the data, leading to unfair predictions and decision supports. Recently, multi-objective hyperparameter optimization has been proposed to search for machine learning models which offer equally Pareto-efficient trade-offs between accuracy and fairness. Although these approaches proved to be more versatile than fairness-aware machine learning algorithms -- which optimize accuracy constrained to some threshold on fairness -- they could drastically increase the energy consumption in the case of large datasets. In this paper we propose FanG-HPO, a Fair and Green Hyperparameter Optimization (HPO) approach based on both multi-objective and multiple information source Bayesian optimization. FanG-HPO uses subsets of the large dataset (aka information sources) to obtain cheap approximations of both accuracy and fairness, and multi-objective Bayesian Optimization to efficiently identify Pareto-efficient machine learning models. Experiments consider two benchmark (fairness) datasets and two machine learning algorithms (XGBoost and Multi-Layer Perceptron), and provide an assessment of FanG-HPO against both fairness-aware machine learning algorithms and hyperparameter optimization via a multi-objective single-source optimization algorithm in BoTorch, a state-of-the-art platform for Bayesian Optimization.