How fair can we go in machine learning? Assessing the boundaries of fairness in decision trees

TL;DR

This paper introduces a multi-objective methodology to explore the trade-offs between fairness and accuracy in decision tree classifiers, providing a Pareto front of optimal solutions to understand the limits of bias mitigation.

Contribution

It presents the first approach to analyze the statistical limits of fairness interventions within decision trees using a multi-objective framework and Pareto optimization.

Findings

The method can improve fairness with minimal increase in error.

Decision trees can be optimized for fairness without significant loss of accuracy.

The approach helps stakeholders understand the boundaries of fairness in machine learning.

Abstract

Fair machine learning works have been focusing on the development of equitable algorithms that address discrimination of certain groups. Yet, many of these fairness-aware approaches aim to obtain a unique solution to the problem, which leads to a poor understanding of the statistical limits of bias mitigation interventions. We present the first methodology that allows to explore those limits within a multi-objective framework that seeks to optimize any measure of accuracy and fairness and provides a Pareto front with the best feasible solutions. In this work, we focus our study on decision tree classifiers since they are widely accepted in machine learning, are easy to interpret and can deal with non-numerical information naturally. We conclude experimentally that our method can optimize decision tree models by being fairer with a small cost of the classification error. We believe that our contribution will help stakeholders of sociotechnical systems to assess how far they can go being fair and accurate, thus serving in the support of enhanced decision making where machine learning is used.