FairNet: Dynamic Fairness Correction without Performance Loss via Contrastive Conditional LoRA

TL;DR

FairNet introduces a dynamic fairness correction framework that selectively applies bias mitigation at the instance level using contrastive learning, improving fairness without sacrificing overall model performance.

Contribution

The paper presents FairNet, a novel method combining a bias detector with contrastive LoRA for dynamic fairness correction adaptable to various sensitive attribute scenarios.

Findings

Enhances worst-group performance without performance loss.

Effectively handles partial or missing sensitive attribute labels.

Validated across vision and language benchmarks.

Abstract

Ensuring fairness in machine learning models is a critical challenge. Existing debiasing methods often compromise performance, rely on static correction strategies, and struggle with data sparsity, particularly within minority groups. Furthermore, their utilization of sensitive attributes is often suboptimal, either depending excessively on complete attribute labeling or disregarding these attributes entirely. To overcome these limitations, we propose FairNet, a novel framework for dynamic, instance-level fairness correction. FairNet integrates a bias detector with conditional low-rank adaptation (LoRA), which enables selective activation of the fairness correction mechanism exclusively for instances identified as biased, and thereby preserve performance on unbiased instances. A key contribution is a new contrastive loss function for training the LoRA module, specifically designed to minimize intra-class representation disparities across different sensitive groups and effectively address underfitting in minority groups. The FairNet framework can flexibly handle scenarios with complete, partial, or entirely absent sensitive attribute labels. Theoretical analysis confirms that, under moderate TPR/FPR for the bias detector, FairNet can enhance the performance of the worst group without diminishing overall model performance, and potentially yield slight performance improvements. Comprehensive empirical evaluations across diverse vision and language benchmarks validate the effectiveness of FairNet.