Multidimensional Scaling, Sammon Mapping, and Isomap: Tutorial and Survey

TL;DR

This paper provides a comprehensive tutorial and survey of multidimensional scaling (MDS), Sammon mapping, and Isomap, covering theory, variants, out-of-sample extensions, and large-scale data embedding techniques.

Contribution

It offers an in-depth review of MDS, Sammon mapping, and Isomap, including recent extensions like out-of-sample embedding and landmark methods for big data.

Findings

Detailed explanation of MDS variants

Introduction of out-of-sample embedding techniques

Discussion of landmark methods for large datasets

Abstract

Multidimensional Scaling (MDS) is one of the first fundamental manifold learning methods. It can be categorized into several methods, i.e., classical MDS, kernel classical MDS, metric MDS, and non-metric MDS. Sammon mapping and Isomap can be considered as special cases of metric MDS and kernel classical MDS, respectively. In this tutorial and survey paper, we review the theory of MDS, Sammon mapping, and Isomap in detail. We explain all the mentioned categories of MDS. Then, Sammon mapping, Isomap, and kernel Isomap are explained. Out-of-sample embedding for MDS and Isomap using eigenfunctions and kernel mapping are introduced. Then, Nystrom approximation and its use in landmark MDS and landmark Isomap are introduced for big data embedding. We also provide some simulations for illustrating the embedding by these methods.