Classification from Pairwise Similarities/Dissimilarities and Unlabeled Data via Empirical Risk Minimization

TL;DR

This paper introduces an empirical risk minimization approach that effectively utilizes pairwise similarities, dissimilarities, and unlabeled data for classification, overcoming limitations of previous methods that only handled similarities.

Contribution

The authors derive an unbiased risk estimator capable of incorporating all pairwise similarities, dissimilarities, and unlabeled data, with theoretical error bounds and practical validation.

Findings

The method provides unbiased risk estimation from pairwise data and unlabeled samples.

Theoretical analysis includes estimation error bounds.

Experimental results demonstrate practical effectiveness.

Abstract

Pairwise similarities and dissimilarities between data points might be easier to obtain than fully labeled data in real-world classification problems, e.g., in privacy-aware situations. To handle such pairwise information, an empirical risk minimization approach has been proposed, giving an unbiased estimator of the classification risk that can be computed only from pairwise similarities and unlabeled data. However, this direction cannot handle pairwise dissimilarities so far. On the other hand, semi-supervised clustering is one of the methods which can use both similarities and dissimilarities. Nevertheless, they typically require strong geometrical assumptions on the data distribution such as the manifold assumption, which may deteriorate the performance. In this paper, we derive an unbiased risk estimator which can handle all of similarities/dissimilarities and unlabeled data. We theoretically establish estimation error bounds and experimentally demonstrate the practical usefulness of our empirical risk minimization method.