A Convergence Rate for Manifold Neural Networks

TL;DR

This paper establishes a convergence rate for manifold neural networks constructed from sampled data, showing the rate depends on the manifold's intrinsic dimension and network parameters, advancing theoretical understanding in geometric deep learning.

Contribution

The paper provides a convergence rate analysis for data-driven manifold neural networks, highlighting dependence on intrinsic dimension and network architecture, extending previous convergence results.

Findings

Convergence rate depends on intrinsic dimension, not ambient dimension.

Rate influenced by network depth and number of filters.

Provides theoretical guarantees for manifold neural network approximation.

Abstract

High-dimensional data arises in numerous applications, and the rapidly developing field of geometric deep learning seeks to develop neural network architectures to analyze such data in non-Euclidean domains, such as graphs and manifolds. Recent work by Z. Wang, L. Ruiz, and A. Ribeiro has introduced a method for constructing manifold neural networks using the spectral decomposition of the Laplace Beltrami operator. Moreover, in this work, the authors provide a numerical scheme for implementing such neural networks when the manifold is unknown and one only has access to finitely many sample points. The authors show that this scheme, which relies upon building a data-driven graph, converges to the continuum limit as the number of sample points tends to infinity. Here, we build upon this result by establishing a rate of convergence that depends on the intrinsic dimension of the manifold but is independent of the ambient dimension. We also discuss how the rate of convergence depends on the depth of the network and the number of filters used in each layer.