Group-based Fair Learning Leads to Counter-intuitive Predictions

TL;DR

This paper reveals that common fairness-enforcing machine learning methods can violate intuitive monotonicity in individual predictions, highlighting potential unintended consequences of fairness constraints.

Contribution

The paper introduces the property of slack-consistency for fairness methods and demonstrates that standard approaches often violate it, even in simple or idealized settings.

Findings

Standard fairness methods violate slack-consistency.

Violations occur regardless of model complexity or optimization accuracy.

A simple theoretical approach can enforce slack-consistency.

Abstract

A number of machine learning (ML) methods have been proposed recently to maximize model predictive accuracy while enforcing notions of group parity or fairness across sub-populations. We propose a desirable property for these procedures, slack-consistency: For any individual, the predictions of the model should be monotonic with respect to allowed slack (i.e., maximum allowed group-parity violation). Such monotonicity can be useful for individuals to understand the impact of enforcing fairness on their predictions. Surprisingly, we find that standard ML methods for enforcing fairness violate this basic property. Moreover, this undesirable behavior arises in situations agnostic to the complexity of the underlying model or approximate optimizations, suggesting that the simple act of incorporating a constraint can lead to drastically unintended behavior in ML. We present a simple theoretical method for enforcing slack-consistency, while encouraging further discussions on the unintended behaviors potentially induced when enforcing group-based parity.