DLME: Deep Local-flatness Manifold Embedding

TL;DR

DLME introduces a novel manifold learning framework that enhances low-dimensional embeddings from high-dimensional data, especially under-sampled data, by leveraging data augmentation, local flatness constraints, and a new loss function, improving downstream task performance.

Contribution

The paper proposes DLME, a new manifold learning method that addresses structural distortion and underconstrained embeddings through data augmentation and local flatness constraints, with theoretical and empirical validation.

Findings

DLME outperforms existing methods on various datasets.

DLME improves embedding quality for classification, clustering, and visualization.

Theoretical analysis supports the effectiveness of the proposed loss.

Abstract

Manifold learning (ML) aims to seek low-dimensional embedding from high-dimensional data. The problem is challenging on real-world datasets, especially with under-sampling data, and we find that previous methods perform poorly in this case. Generally, ML methods first transform input data into a low-dimensional embedding space to maintain the data's geometric structure and subsequently perform downstream tasks therein. The poor local connectivity of under-sampling data in the former step and inappropriate optimization objectives in the latter step leads to two problems: structural distortion and underconstrained embedding. This paper proposes a novel ML framework named Deep Local-flatness Manifold Embedding (DLME) to solve these problems. The proposed DLME constructs semantic manifolds by data augmentation and overcomes the structural distortion problem using a smoothness constrained based on a local flatness assumption about the manifold. To overcome the underconstrained embedding problem, we design a loss and theoretically demonstrate that it leads to a more suitable embedding based on the local flatness. Experiments on three types of datasets (toy, biological, and image) for various downstream tasks (classification, clustering, and visualization) show that our proposed DLME outperforms state-of-the-art ML and contrastive learning methods.