Blessing of Dimensionality for Approximating Sobolev Classes on Manifolds

TL;DR

This paper establishes that the complexity needed to approximate Sobolev functions on manifolds depends solely on intrinsic geometric properties, highlighting the importance of manifold structure in high-dimensional approximation tasks.

Contribution

It provides the first lower bounds on the statistical complexity for approximating Sobolev functions on manifolds, emphasizing the role of intrinsic geometry.

Findings

Lower bounds depend only on manifold curvature, volume, and injectivity radius.

Approximation complexity is independent of ambient dimension.

Results highlight the fundamental difficulty of approximation on manifolds.

Abstract

The manifold hypothesis says that natural high-dimensional data lie on or around a low-dimensional manifold. The recent success of statistical and learning-based methods in very high dimensions empirically supports this hypothesis, suggesting that typical worst-case analysis does not provide practical guarantees. A natural step for analysis is thus to assume the manifold hypothesis and derive bounds that are independent of any ambient dimensions that the data may be embedded in. Theoretical implications in this direction have recently been explored in terms of generalization of ReLU networks and convergence of Langevin methods. In this work, we consider optimal uniform approximations with functions of finite statistical complexity. While upper bounds on uniform approximation exist in the literature using ReLU neural networks, we consider the opposite: lower bounds to quantify the fundamental difficulty of approximation on manifolds. In particular, we demonstrate that the statistical complexity required to approximate a class of bounded Sobolev functions on a compact manifold is bounded from below, and moreover that this bound is dependent only on the intrinsic properties of the manifold, such as curvature, volume, and injectivity radius.