Long-Term Fairness with Unknown Dynamics

TL;DR

This paper introduces a long-term fairness framework in reinforcement learning that balances equity and performance over time, adapting to unknown dynamics with theoretical guarantees.

Contribution

It formalizes long-term fairness in RL, develops an adaptive algorithm with proven bounds, and compares it to existing methods using theoretical and empirical evaluations.

Findings

The proposed algorithm achieves bounds on fairness violations and loss.

It outperforms myopic retraining and deep RL without safety guarantees.

Experiments with real data and game theory models validate effectiveness.

Abstract

While machine learning can myopically reinforce social inequalities, it may also be used to dynamically seek equitable outcomes. In this paper, we formalize long-term fairness in the context of online reinforcement learning. This formulation can accommodate dynamical control objectives, such as driving equity inherent in the state of a population, that cannot be incorporated into static formulations of fairness. We demonstrate that this framing allows an algorithm to adapt to unknown dynamics by sacrificing short-term incentives to drive a classifier-population system towards more desirable equilibria. For the proposed setting, we develop an algorithm that adapts recent work in online learning. We prove that this algorithm achieves simultaneous probabilistic bounds on cumulative loss and cumulative violations of fairness (as statistical regularities between demographic groups). We compare our proposed algorithm to the repeated retraining of myopic classifiers, as a baseline, and to a deep reinforcement learning algorithm that lacks safety guarantees. Our experiments model human populations according to evolutionary game theory and integrate real-world datasets.