Superhuman Fairness

TL;DR

This paper introduces superhuman fairness, a novel approach to fair machine learning that aims to outperform human decisions across multiple performance and fairness metrics by framing it as an imitation learning task.

Contribution

It redefines fair machine learning as an imitation learning problem to achieve better performance and fairness than human decisions, challenging existing trade-off paradigms.

Findings

Superhuman fairness outperforms human decisions on multiple metrics.

Recasting fairness as imitation learning improves decision quality.

The approach benefits scenarios with suboptimal human decisions.

Abstract

The fairness of machine learning-based decisions has become an increasingly important focus in the design of supervised machine learning methods. Most fairness approaches optimize a specified trade-off between performance measure(s) (e.g., accuracy, log loss, or AUC) and fairness metric(s) (e.g., demographic parity, equalized odds). This begs the question: are the right performance-fairness trade-offs being specified? We instead re-cast fair machine learning as an imitation learning task by introducing superhuman fairness, which seeks to simultaneously outperform human decisions on multiple predictive performance and fairness measures. We demonstrate the benefits of this approach given suboptimal decisions.