Improving State-of-the-Art in One-Class Classification by Leveraging Unlabeled Data

TL;DR

This paper compares one-class and positive unlabeled learning methods for binary classification with limited labeled data, highlighting when to use each approach based on unlabeled data reliability, and proposes robust modifications for unreliable data scenarios.

Contribution

It provides an extensive experimental comparison of OC and PU algorithms and introduces modifications to OC algorithms for robustness against unreliable unlabeled data.

Findings

PU algorithms perform better with reliable unlabeled data.

Modified OC algorithms are robust when unlabeled data is unreliable.

Guidelines and statistical tests help determine data reliability.

Abstract

When dealing with binary classification of data with only one labeled class data scientists employ two main approaches, namely One-Class (OC) classification and Positive Unlabeled (PU) learning. The former only learns from labeled positive data, whereas the latter also utilizes unlabeled data to improve the overall performance. Since PU learning utilizes more data, we might be prone to think that when unlabeled data is available, the go-to algorithms should always come from the PU group. However, we find that this is not always the case if unlabeled data is unreliable, i.e. contains limited or biased latent negative data. We perform an extensive experimental study of a wide list of state-of-the-art OC and PU algorithms in various scenarios as far as unlabeled data reliability is concerned. Furthermore, we propose PU modifications of state-of-the-art OC algorithms that are robust to unreliable unlabeled data, as well as a guideline to similarly modify other OC algorithms. Our main practical recommendation is to use state-of-the-art PU algorithms when unlabeled data is reliable and to use the proposed modifications of state-of-the-art OC algorithms otherwise. Additionally, we outline procedures to distinguish the cases of reliable and unreliable unlabeled data using statistical tests.