Sample Complexity of Nonparametric Semi-Supervised Learning

TL;DR

This paper analyzes the sample complexity of nonparametric semi-supervised learning, establishing bounds on labeled samples needed for multiclass classification without parametric assumptions, and introduces algorithms based on bipartite graph matching.

Contribution

It introduces new assumptions based on mixture model mismatch, derives labeled sample complexity bounds for multiclass SSL, and proposes algorithms with experimental validation.

Findings

Labeled sample complexity bound of $oldsymbol{ ext{Ω}(K ext{ log } K)}$ for multiclass SSL.

Near-optimal classifiers can be learned with few labeled samples in nonparametric settings.

Algorithms based on bipartite graph matching outperform majority vote in experiments.

Abstract

We study the sample complexity of semi-supervised learning (SSL) and introduce new assumptions based on the mismatch between a mixture model learned from unlabeled data and the true mixture model induced by the (unknown) class conditional distributions. Under these assumptions, we establish an $\Omega(K\log K)$ labeled sample complexity bound without imposing parametric assumptions, where $K$ is the number of classes. Our results suggest that even in nonparametric settings it is possible to learn a near-optimal classifier using only a few labeled samples. Unlike previous theoretical work which focuses on binary classification, we consider general multiclass classification ($K>2$), which requires solving a difficult permutation learning problem. This permutation defines a classifier whose classification error is controlled by the Wasserstein distance between mixing measures, and we provide finite-sample results characterizing the behaviour of the excess risk of this classifier. Finally, we describe three algorithms for computing these estimators based on a connection to bipartite graph matching, and perform experiments to illustrate the superiority of the MLE over the majority vote estimator.