Consistency of semi-supervised learning algorithms on graphs: Probit and one-hot methods

TL;DR

This paper analyzes the consistency of graph-based semi-supervised learning algorithms, specifically probit and one-hot methods, in the small noise and well-clustered data regime, providing theoretical insights into their optimization functions.

Contribution

It introduces a consistency analysis for graph-based probit and one-hot methods, clarifying the role of the rational function in the optimization process.

Findings

Probit and one-hot methods are consistent under certain data conditions.

The choice of the rational function in the Laplacian significantly affects consistency.

The analysis guides better design of semi-supervised learning algorithms on graphs.

Abstract

Graph-based semi-supervised learning is the problem of propagating labels from a small number of labelled data points to a larger set of unlabelled data. This paper is concerned with the consistency of optimization-based techniques for such problems, in the limit where the labels have small noise and the underlying unlabelled data is well clustered. We study graph-based probit for binary classification, and a natural generalization of this method to multi-class classification using one-hot encoding. The resulting objective function to be optimized comprises the sum of a quadratic form defined through a rational function of the graph Laplacian, involving only the unlabelled data, and a fidelity term involving only the labelled data. The consistency analysis sheds light on the choice of the rational function defining the optimization.