High Performance Out-of-sample Embedding Techniques for Multidimensional Scaling

TL;DR

This paper introduces out-of-sample embedding methods to scale multidimensional scaling for large datasets, using optimization and neural networks, enabling efficient dimension reduction for streaming and static data.

Contribution

It presents two novel out-of-sample embedding techniques for MDS, significantly improving scalability and efficiency for large-scale data analysis.

Findings

Neural network-based OSE outperforms optimization-based OSE in efficiency.

Both OSE methods enable large-scale MDS with reduced memory and computation.

The methods are suitable for streaming and static datasets.

Abstract

The recent rapid growth of the dimension of many datasets means that many approaches to dimension reduction (DR) have gained significant attention. High-performance DR algorithms are required to make data analysis feasible for big and fast data sets. However, many traditional DR techniques are challenged by truly large data sets. In particular multidimensional scaling (MDS) does not scale well. MDS is a popular group of DR techniques because it can perform DR on data where the only input is a dissimilarity function. However, common approaches are at least quadratic in memory and computation and, hence, prohibitive for large-scale data. We propose an out-of-sample embedding (OSE) solution to extend the MDS algorithm for large-scale data utilising the embedding of only a subset of the given data. We present two OSE techniques: the first based on an optimisation approach and the second based on a neural network model. With a minor trade-off in the approximation, the out-of-sample techniques can process large-scale data with reasonable computation and memory requirements. While both methods perform well, the neural network model outperforms the optimisation approach of the OSE solution in terms of efficiency. OSE has the dual benefit that it allows fast DR on streaming datasets as well as static databases.