Positive-Unlabeled Learning with Non-Negative Risk Estimator

TL;DR

This paper introduces a non-negative risk estimator for positive-unlabeled learning that enhances robustness against overfitting, enabling the effective use of flexible models like deep neural networks with limited positive data.

Contribution

It proposes a novel non-negative risk estimator for PU learning that reduces overfitting and allows the use of highly flexible models, with theoretical analysis and empirical validation.

Findings

Fixes overfitting issues of unbiased PU learning methods

Enables training with deep neural networks on limited positive data

Provides theoretical bounds and analysis of the estimator

Abstract

From only positive (P) and unlabeled (U) data, a binary classifier could be trained with PU learning, in which the state of the art is unbiased PU learning. However, if its model is very flexible, empirical risks on training data will go negative, and we will suffer from serious overfitting. In this paper, we propose a non-negative risk estimator for PU learning: when getting minimized, it is more robust against overfitting, and thus we are able to use very flexible models (such as deep neural networks) given limited P data. Moreover, we analyze the bias, consistency, and mean-squared-error reduction of the proposed risk estimator, and bound the estimation error of the resulting empirical risk minimizer. Experiments demonstrate that our risk estimator fixes the overfitting problem of its unbiased counterparts.