Semi-Supervised Kernel PCA

TL;DR

This paper introduces three semi-supervised extensions of Kernel PCA that leverage limited label information to improve data representation, combining discriminant, regression, and reweighted loss approaches with theoretical and experimental validation.

Contribution

It proposes three novel semi-supervised Kernel PCA methods—MV-KPCA, LSKPCA, and LR-KPCA—that incorporate label information to enhance feature extraction.

Findings

MV-KPCA penalizes within-class variance.

LSKPCA combines least squares regression with KPCA.

LR-KPCA uses reweighted sigmoid loss for labeled points.

Abstract

We present three generalisations of Kernel Principal Components Analysis (KPCA) which incorporate knowledge of the class labels of a subset of the data points. The first, MV-KPCA, penalises within class variances similar to Fisher discriminant analysis. The second, LSKPCA is a hybrid of least squares regression and kernel PCA. The final LR-KPCA is an iteratively reweighted version of the previous which achieves a sigmoid loss function on the labeled points. We provide a theoretical risk bound as well as illustrative experiments on real and toy data sets.