Revisiting Fairness Impossibility with Endogenous Behavior

TL;DR

This paper explores how strategic responses to classification influence fairness tradeoffs, revealing that fairness criteria interact with behavioral incentives, leading to new considerations in algorithmic fairness.

Contribution

It introduces a two-stage design where classifiers standardize performance and then adjust stakes to induce comparable behaviors across groups, highlighting new fairness tradeoffs.

Findings

Incompatibility between error-rate balance and predictive parity can be resolved with different treatment of groups.

Adjusting stakes can induce similar behavioral patterns across groups, affecting fairness outcomes.

Fairness in strategic settings depends on how consequences are designed, not just algorithmic decisions.

Abstract

In many real-world settings, institutions can and do adjust the consequences attached to algorithmic classification decisions, such as the size of fines, sentence lengths, or benefit levels. We refer to these consequences as the stakes associated with classification. These stakes can give rise to behavioral responses to classification, as people adjust their actions in anticipation of how they will be classified. Much of the algorithmic fairness literature evaluates classification outcomes while holding behavior fixed, treating behavioral differences across groups as exogenous features of the environment. Under this assumption, the stakes of classification play no role in shaping outcomes. We revisit classic impossibility results in algorithmic fairness in a setting where people respond strategically to classification. We show that, in this environment, the well-known incompatibility between error-rate balance and predictive parity disappears, but only by potentially introducing a qualitatively different form of unequal treatment. Concretely, we construct a two-stage design in which a classifier first standardizes its statistical performance across groups, and then adjusts stakes so as to induce comparable patterns of behavior. This requires treating groups differently in the consequences attached to identical classification decisions. Our results demonstrate that fairness in strategic settings cannot be assessed solely by how algorithms map data into decisions. Rather, our analysis treats the human consequences of classification as primary design variables, introduces normative criteria governing their use, and shows that their interaction with statistical fairness criteria generates qualitatively new tradeoffs. Our aim is to make these tradeoffs precise and explicit.