Fair Multi-Exit Framework for Facial Attribute Classification

TL;DR

This paper introduces a fairness-oriented multi-exit neural network framework for facial attribute classification, enabling early exit for high-confidence instances to improve fairness without sacrificing accuracy.

Contribution

It extends multi-exit neural network frameworks to prioritize fairness, allowing early exit based on confidence and fairness metrics, applicable to existing fairness-aware models.

Findings

Significant fairness improvements over state-of-the-art methods.

Effective early exit mechanism based on confidence levels.

Applicable to various existing fairness frameworks.

Abstract

Fairness has become increasingly pivotal in facial recognition. Without bias mitigation, deploying unfair AI would harm the interest of the underprivileged population. In this paper, we observe that though the higher accuracy that features from the deeper layer of a neural networks generally offer, fairness conditions deteriorate as we extract features from deeper layers. This phenomenon motivates us to extend the concept of multi-exit framework. Unlike existing works mainly focusing on accuracy, our multi-exit framework is fairness-oriented, where the internal classifiers are trained to be more accurate and fairer. During inference, any instance with high confidence from an internal classifier is allowed to exit early. Moreover, our framework can be applied to most existing fairness-aware frameworks. Experiment results show that the proposed framework can largely improve the fairness condition over the state-of-the-art in CelebA and UTK Face datasets.