No Pressure! Addressing the Problem of Local Minima in Manifold Learning Algorithms

TL;DR

This paper introduces a method to improve manifold learning algorithms by temporarily adding an extra dimension to escape local minima, enhancing embedding quality in high-dimensional data visualization.

Contribution

It proposes a novel extension that identifies pressured points and allows them to use an additional dimension to escape poor local minima, improving existing manifold learning methods.

Findings

Method effectively reduces the objective function after local minima are reached.

Enhances the quality of embeddings in high-dimensional data visualization.

Applicable to multiple manifold learning algorithms.

Abstract

Nonlinear embedding manifold learning methods provide invaluable visual insights into the structure of high-dimensional data. However, due to a complicated nonconvex objective function, these methods can easily get stuck in local minima and their embedding quality can be poor. We propose a natural extension to several manifold learning methods aimed at identifying pressured points, i.e. points stuck in poor local minima and have poor embedding quality. We show that the objective function can be decreased by temporarily allowing these points to make use of an extra dimension in the embedding space. Our method is able to improve the objective function value of existing methods even after they get stuck in a poor local minimum.