SimFair: A Unified Framework for Fairness-Aware Multi-Label Classification

TL;DR

This paper introduces SimFair, a unified fairness framework for multi-label classification that extends existing fairness notions and improves stability and performance on real-world datasets.

Contribution

It extends fairness notions to multi-label classification and proposes SimFair, a new framework that unifies fairness criteria and enhances stability in predictions.

Findings

SimFair outperforms existing methods on real-world datasets.

The framework effectively unifies Demographic Parity and Equalized Opportunity.

It improves fairness stability in multi-label classification tasks.

Abstract

Recent years have witnessed increasing concerns towards unfair decisions made by machine learning algorithms. To improve fairness in model decisions, various fairness notions have been proposed and many fairness-aware methods are developed. However, most of existing definitions and methods focus only on single-label classification. Fairness for multi-label classification, where each instance is associated with more than one labels, is still yet to establish. To fill this gap, we study fairness-aware multi-label classification in this paper. We start by extending Demographic Parity (DP) and Equalized Opportunity (EOp), two popular fairness notions, to multi-label classification scenarios. Through a systematic study, we show that on multi-label data, because of unevenly distributed labels, EOp usually fails to construct a reliable estimate on labels with few instances. We then propose a new framework named Similarity $s$-induced Fairness ($s_\gamma$-SimFair). This new framework utilizes data that have similar labels when estimating fairness on a particular label group for better stability, and can unify DP and EOp. Theoretical analysis and experimental results on real-world datasets together demonstrate the advantage of over existing methods $s_\gamma$-SimFair on multi-label classification tasks.