Fairness-aware Model-agnostic Positive and Unlabeled Learning

TL;DR

This paper introduces FairPUL, a model-agnostic post-processing framework for fairness-aware positive and unlabeled learning, addressing bias in high-stakes decision-making scenarios with positive-unlabeled data.

Contribution

It proposes a novel fairness-aware PUL method that ensures similar true and false positive rates across populations, with proven statistical consistency and superior performance.

Findings

Outperforms state-of-the-art in PUL and fair classification.

Ensures similar true and false positive rates across groups.

Proven to be statistically consistent.

Abstract

With the increasing application of machine learning in high-stake decision-making problems, potential algorithmic bias towards people from certain social groups poses negative impacts on individuals and our society at large. In the real-world scenario, many such problems involve positive and unlabeled data such as medical diagnosis, criminal risk assessment and recommender systems. For instance, in medical diagnosis, only the diagnosed diseases will be recorded (positive) while others will not (unlabeled). Despite the large amount of existing work on fairness-aware machine learning in the (semi-)supervised and unsupervised settings, the fairness issue is largely under-explored in the aforementioned Positive and Unlabeled Learning (PUL) context, where it is usually more severe. In this paper, to alleviate this tension, we propose a fairness-aware PUL method named FairPUL. In particular, for binary classification over individuals from two populations, we aim to achieve similar true positive rates and false positive rates in both populations as our fairness metric. Based on the analysis of the optimal fair classifier for PUL, we design a model-agnostic post-processing framework, leveraging both the positive examples and unlabeled ones. Our framework is proven to be statistically consistent in terms of both the classification error and the fairness metric. Experiments on the synthetic and real-world data sets demonstrate that our framework outperforms state-of-the-art in both PUL and fair classification.