A scale-based approach to finding effective dimensionality in manifold learning

TL;DR

This paper introduces a scale-based method for accurately determining the intrinsic dimensionality of low-dimensional manifolds within high-dimensional data, effectively handling noise without prior knowledge.

Contribution

It presents a novel scale space approach that identifies effective dimensionality across all scales, outperforming existing methods especially in noisy conditions.

Findings

Better performance in noisy data scenarios

Computationally efficient method

Effective across all scales without prior knowledge

Abstract

The discovering of low-dimensional manifolds in high-dimensional data is one of the main goals in manifold learning. We propose a new approach to identify the effective dimension (intrinsic dimension) of low-dimensional manifolds. The scale space viewpoint is the key to our approach enabling us to meet the challenge of noisy data. Our approach finds the effective dimensionality of the data over all scale without any prior knowledge. It has better performance compared with other methods especially in the presence of relatively large noise and is computationally efficient.