Accounting for Model Uncertainty in Algorithmic Discrimination

TL;DR

This paper advocates for fairness in algorithmic decision making by focusing on equalizing errors caused specifically by model uncertainty, proposing scalable methods to achieve this goal efficiently.

Contribution

It introduces a novel approach to fairness that isolates model uncertainty errors, connects predictive multiplicity to uncertainty, and offers scalable algorithms to mitigate these errors.

Findings

Methods are up to four orders of magnitude faster than existing techniques.

Proposed classifiers effectively equalize errors due to model uncertainty.

Empirical results on synthetic and real datasets validate the approach.

Abstract

Traditional approaches to ensure group fairness in algorithmic decision making aim to equalize ``total'' error rates for different subgroups in the population. In contrast, we argue that the fairness approaches should instead focus only on equalizing errors arising due to model uncertainty (a.k.a epistemic uncertainty), caused due to lack of knowledge about the best model or due to lack of data. In other words, our proposal calls for ignoring the errors that occur due to uncertainty inherent in the data, i.e., aleatoric uncertainty. We draw a connection between predictive multiplicity and model uncertainty and argue that the techniques from predictive multiplicity could be used to identify errors made due to model uncertainty. We propose scalable convex proxies to come up with classifiers that exhibit predictive multiplicity and empirically show that our methods are comparable in performance and up to four orders of magnitude faster than the current state-of-the-art. We further propose methods to achieve our goal of equalizing group error rates arising due to model uncertainty in algorithmic decision making and demonstrate the effectiveness of these methods using synthetic and real-world datasets.