Toward a better trade-off between performance and fairness with kernel-based distribution matching

TL;DR

This paper introduces a MinDiff framework with kernel-based dependency tests to balance classifier performance and fairness, demonstrating improvements on academic and industrial datasets.

Contribution

It proposes a novel regularization method for fairness in classifiers using kernel-based distribution matching, with comprehensive analysis and real-world applications.

Findings

Improved fairness-performance trade-offs on academic datasets.

Effective application of kernel-based regularization in industrial systems.

Enhanced understanding of Pareto frontiers for fairness and accuracy.

Abstract

As recent literature has demonstrated how classifiers often carry unintended biases toward some subgroups, deploying machine learned models to users demands careful consideration of the social consequences. How should we address this problem in a real-world system? How should we balance core performance and fairness metrics? In this paper, we introduce a MinDiff framework for regularizing classifiers toward different fairness metrics and analyze a technique with kernel-based statistical dependency tests. We run a thorough study on an academic dataset to compare the Pareto frontier achieved by different regularization approaches, and apply our kernel-based method to two large-scale industrial systems demonstrating real-world improvements.