Robust Optimization for Fairness with Noisy Protected Groups

TL;DR

This paper addresses the challenge of ensuring fairness in machine learning when protected group labels are noisy, proposing robust optimization methods that guarantee fairness on true groups while maintaining training performance.

Contribution

It introduces two robust optimization approaches that ensure fairness on true protected groups despite noisy labels, with theoretical guarantees and empirical validation.

Findings

Robust methods outperform naive approaches in fairness guarantees.

Theoretical convergence to optimal solutions is established.

Empirical case studies demonstrate improved true group fairness.

Abstract

Many existing fairness criteria for machine learning involve equalizing some metric across protected groups such as race or gender. However, practitioners trying to audit or enforce such group-based criteria can easily face the problem of noisy or biased protected group information. First, we study the consequences of naively relying on noisy protected group labels: we provide an upper bound on the fairness violations on the true groups G when the fairness criteria are satisfied on noisy groups $\hat{G}$. Second, we introduce two new approaches using robust optimization that, unlike the naive approach of only relying on $\hat{G}$, are guaranteed to satisfy fairness criteria on the true protected groups G while minimizing a training objective. We provide theoretical guarantees that one such approach converges to an optimal feasible solution. Using two case studies, we show empirically that the robust approaches achieve better true group fairness guarantees than the naive approach.