FairlyUncertain: A Comprehensive Benchmark of Uncertainty in Algorithmic Fairness

TL;DR

FairlyUncertain introduces a benchmark for evaluating how well uncertainty estimates in fairness models are consistent and calibrated, revealing insights into their impact on fairness across various datasets and tasks.

Contribution

It provides the first comprehensive axiomatic benchmark for assessing uncertainty in fairness, including a simple method for binary settings and analysis of uncertainty's role in fairness.

Findings

A simple uncertainty estimation method outperforms prior work in binary fairness.

Uncertainty abstention reduces error but does not fix demographic outcome imbalances.

Calibrated uncertainty improves fairness in regression tasks without additional fairness interventions.

Abstract

Fair predictive algorithms hinge on both equality and trust, yet inherent uncertainty in real-world data challenges our ability to make consistent, fair, and calibrated decisions. While fairly managing predictive error has been extensively explored, some recent work has begun to address the challenge of fairly accounting for irreducible prediction uncertainty. However, a clear taxonomy and well-specified objectives for integrating uncertainty into fairness remains undefined. We address this gap by introducing FairlyUncertain, an axiomatic benchmark for evaluating uncertainty estimates in fairness. Our benchmark posits that fair predictive uncertainty estimates should be consistent across learning pipelines and calibrated to observed randomness. Through extensive experiments on ten popular fairness datasets, our evaluation reveals: (1) A theoretically justified and simple method for estimating uncertainty in binary settings is more consistent and calibrated than prior work; (2) Abstaining from binary predictions, even with improved uncertainty estimates, reduces error but does not alleviate outcome imbalances between demographic groups; (3) Incorporating consistent and calibrated uncertainty estimates in regression tasks improves fairness without any explicit fairness interventions. Additionally, our benchmark package is designed to be extensible and open-source, to grow with the field. By providing a standardized framework for assessing the interplay between uncertainty and fairness, FairlyUncertain paves the way for more equitable and trustworthy machine learning practices.