Semi-supervised learning by search of optimal target vector

TL;DR

This paper introduces a semi-supervised learning estimator based on the optimal target vector concept, which aligns with the first kernel principal component when labels are scarce, enabling dimensionality reduction and semi-supervised classification.

Contribution

The paper proposes a novel semi-supervised learning estimator using the optimal target vector, bridging kernel PCA and regression/classification tasks with partially labeled data.

Findings

The estimator aligns with the first kernel principal component as labeled data diminishes.

It effectively performs dimensionality reduction with limited labels.

It enables semi-supervised regression and classification in transductive settings.

Abstract

We introduce a semi-supervised learning estimator which tends to the first kernel principal component as the number of labelled points vanishes. Our approach is based on the notion of optimal target vector, which is defined as follows. Given an input data-set of ${\bf x}$ values, the optimal target vector $\mathbf{y}$ is such that treating it as the target and using kernel ridge regression to model the dependency of $y$ on ${\bf x}$, the training error achieves its minimum value. For an unlabeled data set, the first kernel principal component is the optimal vector. In the case one is given a partially labeled data set, still one may look for the optimal target vector minimizing the training error. We use this new estimator in two directions. As a substitute of kernel principal component analysis, in the case one has some labeled data, to produce dimensionality reduction. Second, to develop a semi-supervised regression and classification algorithm for transductive inference. We show application of the proposed method in both directions.