Fair Robust Active Learning by Joint Inconsistency

TL;DR

This paper introduces FRAL, a novel active learning framework that enhances fairness and robustness in machine learning models by leveraging joint inconsistency measures, achieving superior fairness with limited labels.

Contribution

The paper proposes a new joint inconsistency method within FRAL to improve fairness and robustness efficiently, addressing data imbalance and computational challenges in adversarial settings.

Findings

Outperforms existing methods in fairness under adversarial attacks

Effectively mitigates class imbalance with inconsistency-based data selection

Achieves superior fairness and robustness on diverse datasets

Abstract

Fairness and robustness play vital roles in trustworthy machine learning. Observing safety-critical needs in various annotation-expensive vision applications, we introduce a novel learning framework, Fair Robust Active Learning (FRAL), generalizing conventional active learning to fair and adversarial robust scenarios. This framework allows us to achieve standard and robust minimax fairness with limited acquired labels. In FRAL, we then observe existing fairness-aware data selection strategies suffer from either ineffectiveness under severe data imbalance or inefficiency due to huge computations of adversarial training. To address these two problems, we develop a novel Joint INconsistency (JIN) method exploiting prediction inconsistencies between benign and adversarial inputs as well as between standard and robust models. These two inconsistencies can be used to identify potential fairness gains and data imbalance mitigations. Thus, by performing label acquisition with our inconsistency-based ranking metrics, we can alleviate the class imbalance issue and enhance minimax fairness with limited computation. Extensive experiments on diverse datasets and sensitive groups demonstrate that our method obtains the best results in standard and robust fairness under white-box PGD attacks compared with existing active data selection baselines.