The Fairness-Accuracy Pareto Front

TL;DR

This paper introduces formal tools to analyze and optimize the trade-off between fairness and accuracy in machine learning classifiers, emphasizing Pareto optimality and proposing the Chebyshev scalarization scheme as a superior method.

Contribution

It applies Pareto front analysis to fairness-accuracy trade-offs and demonstrates the limitations of existing methods, proposing the Chebyshev scalarization scheme for better Pareto optimal solutions.

Findings

Many existing fairness algorithms use linear scalarization, which has limitations.

Chebyshev scalarization is theoretically superior for Pareto optimization.

Chebyshev scheme is computationally comparable to linear methods.

Abstract

Algorithmic fairness seeks to identify and correct sources of bias in machine learning algorithms. Confoundingly, ensuring fairness often comes at the cost of accuracy. We provide formal tools in this work for reconciling this fundamental tension in algorithm fairness. Specifically, we put to use the concept of Pareto optimality from multi-objective optimization and seek the fairness-accuracy Pareto front of a neural network classifier. We demonstrate that many existing algorithmic fairness methods are performing the so-called linear scalarization scheme which has severe limitations in recovering Pareto optimal solutions. We instead apply the Chebyshev scalarization scheme which is provably superior theoretically and no more computationally burdensome at recovering Pareto optimal solutions compared to the linear scheme.