Metric Space Spread, Intrinsic Dimension and the Manifold Hypothesis

TL;DR

This paper introduces a practical method to estimate the intrinsic dimension of data by leveraging the spread dimension of metric spaces, with applications in manifold learning and biodiversity quantification.

Contribution

It develops a novel approach to estimate the topological dimension of manifolds using spread dimension, supported by empirical validation on real and synthetic datasets.

Findings

Spread dimension accurately estimates manifold dimension

Method performs well on real-world data

Provides a theoretical basis for intrinsic dimension estimation

Abstract

The concepts of spread and spread dimension of a metric space were introduced by Willerton in the context of quantifying biodiversity of ecosystems. This paper develops practical applications of spread dimension in the context of machine learning and manifold learning; we show that the topological dimension of a Riemannian manifold can be accurately estimated by computing the spread dimension of a finite subset. These results are presented as the theoretical basis for a novel method of estimating the intrinsic dimension of data. The practical applications of this method are demonstrated with empirical computations using real and synthetic data.