Improving Fairness with Ensemble Combination: Margin-Dependent Bounds

TL;DR

This paper introduces a new fairness measure called discriminative risk that captures both individual and group fairness, and provides theoretical bounds showing fairness can be improved through ensemble methods with margin-dependent guarantees.

Contribution

It proposes a novel fairness measure and establishes the first theoretical bounds for fairness improvement via ensemble combination, applicable to both binary and multi-class classification.

Findings

Discriminative risk effectively captures both fairness aspects.

Theoretical bounds demonstrate potential for fairness enhancement through ensemble methods.

Experimental results validate the effectiveness of the proposed measure and pruning techniques.

Abstract

The concern about hidden discrimination in machine learning models is growing, as their widespread real-world applications increasingly impact human lives. Various techniques, including commonly used group fairness measures and several fairness-aware ensemble-based methods, have been developed to enhance fairness. However, existing fairness measures typically focus on only one aspect -- either group or individual fairness, and the compatibility difficulty among these measures indicates a possibility of remaining biases even when one of them is satisfied. Moreover, existing mechanisms to boost fairness usually present empirical results to show validity, yet few of them discuss whether fairness can be boosted with certain theoretical guarantees. To address these issues, we propose a fairness quality measure named `discriminative risk' by only perturbing protected attributes in instances, to express both individual and group fairness aspects. Furthermore, we investigate its properties and establish the first- and second-order oracle bounds and their relaxations, which show that fairness is possibly improved via ensemble combination with margin-dependent bounds. The analysis is suitable for both binary and multi-class classification. A few ensemble pruning methods are also proposed to utilise our proposed measure and obtain both accurate and fair sub-ensembles; comprehensive experiments are conducted to evaluate the effectiveness of the proposed fairness measure and pruning methods.