Gaussian Process Regression for Out-of-Sample Extension

TL;DR

This paper introduces a Bayesian Gaussian Process Regression method for out-of-sample extension in manifold learning, enabling efficient embedding of new data points and providing abnormality measures, independent of the original manifold learning technique.

Contribution

It presents a novel non-parametric approach that generalizes the Nystrom extension and improves out-of-sample embedding in manifold learning.

Findings

The method effectively embeds unseen data points.

It offers a measure for abnormality detection.

Experimental results outperform existing methods.

Abstract

Manifold learning methods are useful for high dimensional data analysis. Many of the existing methods produce a low dimensional representation that attempts to describe the intrinsic geometric structure of the original data. Typically, this process is computationally expensive and the produced embedding is limited to the training data. In many real life scenarios, the ability to produce embedding of unseen samples is essential. In this paper we propose a Bayesian non-parametric approach for out-of-sample extension. The method is based on Gaussian Process Regression and independent of the manifold learning algorithm. Additionally, the method naturally provides a measure for the degree of abnormality for a newly arrived data point that did not participate in the training process. We derive the mathematical connection between the proposed method and the Nystrom extension and show that the latter is a special case of the former. We present extensive experimental results that demonstrate the performance of the proposed method and compare it to other existing out-of-sample extension methods.