Fair-Net: A Network Architecture For Reducing Performance Disparity Between Identifiable Sub-Populations

TL;DR

Fair-Net is a neural network architecture designed to reduce performance disparities across sub-populations in imbalanced datasets, improving accuracy and calibration for under-represented groups.

Contribution

It introduces a simple yet effective branched multitask architecture that enhances fairness and calibration across sub-populations in class imbalanced data.

Findings

Reduces performance disparity between gender and racial groups.

Improves classification accuracy on minority sub-populations.

Enhances probability calibration across sub-populations.

Abstract

In real world datasets, particular groups are under-represented, much rarer than others, and machine learning classifiers will often preform worse on under-represented populations. This problem is aggravated across many domains where datasets are class imbalanced, with a minority class far rarer than the majority class. Naive approaches to handle under-representation and class imbalance include training sub-population specific classifiers that handle class imbalance or training a global classifier that overlooks sub-population disparities and aims to achieve high overall accuracy by handling class imbalance. In this study, we find that these approaches are vulnerable in class imbalanced datasets with minority sub-populations. We introduced Fair-Net, a branched multitask neural network architecture that improves both classification accuracy and probability calibration across identifiable sub-populations in class imbalanced datasets. Fair-Nets is a straightforward extension to the output layer and error function of a network, so can be incorporated in far more complex architectures. Empirical studies with three real world benchmark datasets demonstrate that Fair-Net improves classification and calibration performance, substantially reducing performance disparity between gender and racial sub-populations.