Out-of-sample generalizations for supervised manifold learning for classification

TL;DR

This paper introduces a semi-supervised out-of-sample extension method for supervised manifold learning in classification, using RBF interpolation to improve generalization to new data points.

Contribution

It proposes a novel RBF-based interpolation algorithm that jointly estimates class labels and interpolation parameters for better out-of-sample generalization.

Findings

Effective on face and object image datasets

Improves classification accuracy for manifold-modeled data

Demonstrates potential for real-world applications

Abstract

Supervised manifold learning methods for data classification map data samples residing in a high-dimensional ambient space to a lower-dimensional domain in a structure-preserving way, while enhancing the separation between different classes in the learned embedding. Most nonlinear supervised manifold learning methods compute the embedding of the manifolds only at the initially available training points, while the generalization of the embedding to novel points, known as the out-of-sample extension problem in manifold learning, becomes especially important in classification applications. In this work, we propose a semi-supervised method for building an interpolation function that provides an out-of-sample extension for general supervised manifold learning algorithms studied in the context of classification. The proposed algorithm computes a radial basis function (RBF) interpolator that minimizes an objective function consisting of the total embedding error of unlabeled test samples, defined as their distance to the embeddings of the manifolds of their own class, as well as a regularization term that controls the smoothness of the interpolation function in a direction-dependent way. The class labels of test data and the interpolation function parameters are estimated jointly with a progressive procedure. Experimental results on face and object images demonstrate the potential of the proposed out-of-sample extension algorithm for the classification of manifold-modeled data sets.