Scalable Out-of-Sample Extension of Graph Embeddings Using Deep Neural Networks

TL;DR

This paper introduces a deep neural network-based method for efficiently extending graph embeddings to new data points, offering comparable or better accuracy with significantly reduced computational costs.

Contribution

It proposes a parametric out-of-sample extension technique using DNNs that generalizes graph embeddings more efficiently than traditional methods.

Findings

DNNs can match or outperform traditional extension methods in fidelity.

The approach significantly reduces computation time at test time.

Unsupervised pretraining enhances DNN optimization for larger models.

Abstract

Several popular graph embedding techniques for representation learning and dimensionality reduction rely on performing computationally expensive eigendecompositions to derive a nonlinear transformation of the input data space. The resulting eigenvectors encode the embedding coordinates for the training samples only, and so the embedding of novel data samples requires further costly computation. In this paper, we present a method for the out-of-sample extension of graph embeddings using deep neural networks (DNN) to parametrically approximate these nonlinear maps. Compared with traditional nonparametric out-of-sample extension methods, we demonstrate that the DNNs can generalize with equal or better fidelity and require orders of magnitude less computation at test time. Moreover, we find that unsupervised pretraining of the DNNs improves optimization for larger network sizes, thus removing sensitivity to model selection.