Enhancing Fairness of Visual Attribute Predictors

TL;DR

This paper introduces fairness-aware regularization losses for deep neural networks to mitigate bias in visual attribute prediction tasks, demonstrating improved fairness without sacrificing accuracy.

Contribution

It proposes novel fairness regularization losses based on Demographic Parity, Equalized Odds, and Intersection-over-Union, integrated into end-to-end training for bias mitigation in visual attribute predictors.

Findings

Improved fairness metrics on facial and medical image datasets

Maintained high classification performance while reducing bias

First to incorporate these fairness losses in end-to-end visual attribute prediction training

Abstract

The performance of deep neural networks for image recognition tasks such as predicting a smiling face is known to degrade with under-represented classes of sensitive attributes. We address this problem by introducing fairness-aware regularization losses based on batch estimates of Demographic Parity, Equalized Odds, and a novel Intersection-over-Union measure. The experiments performed on facial and medical images from CelebA, UTKFace, and the SIIM-ISIC melanoma classification challenge show the effectiveness of our proposed fairness losses for bias mitigation as they improve model fairness while maintaining high classification performance. To the best of our knowledge, our work is the first attempt to incorporate these types of losses in an end-to-end training scheme for mitigating biases of visual attribute predictors. Our code is available at https://github.com/nish03/FVAP.