Preserving AUC Fairness in Learning with Noisy Protected Groups

TL;DR

This paper introduces a novel robust method for maintaining fairness in AUC optimization when protected group labels are noisy, using distributionally robust optimization with theoretical guarantees.

Contribution

It presents the first approach to ensure AUC fairness under noisy protected groups, addressing a critical gap in existing fairness mitigation techniques.

Findings

Outperforms state-of-the-art methods in preserving AUC fairness

Effective on both tabular and image datasets

Provides theoretical guarantees for fairness under noise

Abstract

The Area Under the ROC Curve (AUC) is a key metric for classification, especially under class imbalance, with growing research focus on optimizing AUC over accuracy in applications like medical image analysis and deepfake detection. This leads to fairness in AUC optimization becoming crucial as biases can impact protected groups. While various fairness mitigation techniques exist, fairness considerations in AUC optimization remain in their early stages, with most research focusing on improving AUC fairness under the assumption of clean protected groups. However, these studies often overlook the impact of noisy protected groups, leading to fairness violations in practice. To address this, we propose the first robust AUC fairness approach under noisy protected groups with fairness theoretical guarantees using distributionally robust optimization. Extensive experiments on tabular and image datasets show that our method outperforms state-of-the-art approaches in preserving AUC fairness. The code is in https://github.com/Purdue-M2/AUC_Fairness_with_Noisy_Groups.