CosFairNet:A Parameter-Space based Approach for Bias Free Learning

TL;DR

CosFairNet introduces a novel parameter-space approach that trains dual models to explicitly prevent bias propagation across neural network layers, improving fairness and accuracy in biased data scenarios.

Contribution

This paper presents a new method that directly addresses bias in neural network parameters by training two models with layer-wise similarity constraints, unlike prior feature or sample space techniques.

Findings

Enhanced classification accuracy on biased datasets

Effective bias mitigation across various bias types

Robustness to different bias levels and sample sizes

Abstract

Deep neural networks trained on biased data often inadvertently learn unintended inference rules, particularly when labels are strongly correlated with biased features. Existing bias mitigation methods typically involve either a) predefining bias types and enforcing them as prior knowledge or b) reweighting training samples to emphasize bias-conflicting samples over bias-aligned samples. However, both strategies address bias indirectly in the feature or sample space, with no control over learned weights, making it difficult to control the bias propagation across different layers. Based on this observation, we introduce a novel approach to address bias directly in the model's parameter space, preventing its propagation across layers. Our method involves training two models: a bias model for biased features and a debias model for unbiased details, guided by the bias model. We enforce dissimilarity in the debias model's later layers and similarity in its initial layers with the bias model, ensuring it learns unbiased low-level features without adopting biased high-level abstractions. By incorporating this explicit constraint during training, our approach shows enhanced classification accuracy and debiasing effectiveness across various synthetic and real-world datasets of different sizes. Moreover, the proposed method demonstrates robustness across different bias types and percentages of biased samples in the training data. The code is available at: https://visdomlab.github.io/CosFairNet/