Adaptive Semisupervised Inference

TL;DR

This paper introduces an adaptive semisupervised learning method that employs a density-sensitive kernel, improving performance under certain conditions and ensuring it never performs worse than supervised methods, even when assumptions fail.

Contribution

It proposes a new semisupervised learner using a density-sensitive kernel and an adaptive parameter to optimize performance across different data conditions.

Findings

Outperforms supervised learners when the density support set has a small condition number.

Adapts to the degree of semi-supervisedness using data-dependent parameter tuning.

Guarantees no worse performance than supervised learning regardless of assumption validity.

Abstract

Semisupervised methods inevitably invoke some assumption that links the marginal distribution of the features to the regression function of the label. Most commonly, the cluster or manifold assumptions are used which imply that the regression function is smooth over high-density clusters or manifolds supporting the data. A generalization of these assumptions is that the regression function is smooth with respect to some density sensitive distance. This motivates the use of a density based metric for semisupervised learning. We analyze this setting and make the following contributions - (a) we propose a semi-supervised learner that uses a density-sensitive kernel and show that it provides better performance than any supervised learner if the density support set has a small condition number and (b) we show that it is possible to adapt to the degree of semi-supervisedness using data-dependent choice of a parameter that controls sensitivity of the distance metric to the density. This ensures that the semisupervised learner never performs worse than a supervised learner even if the assumptions fail to hold.