Generating Rectifiable Measures through Neural Networks

TL;DR

This paper proves that countably m-rectifiable measures can be approximated by neural networks with quantized weights in Wasserstein distance, with approximation complexity depending on the rectifiability parameter m, not the ambient dimension.

Contribution

It establishes universal approximation results for rectifiable measures using neural networks, improving previous bounds and extending to countably rectifiable measures under decay conditions.

Findings

Neural networks can approximate m-rectifiable measures with small Wasserstein error.

The number of networks needed scales with the rectifiability parameter m, not the ambient dimension.

Extension of results to countably m-rectifiable measures with exponential decay conditions.

Abstract

We derive universal approximation results for the class of (countably) $m$-rectifiable measures. Specifically, we prove that $m$-rectifiable measures can be approximated as push-forwards of the one-dimensional Lebesgue measure on $[0,1]$ using ReLU neural networks with arbitrarily small approximation error in terms of Wasserstein distance. What is more, the weights in the networks under consideration are quantized and bounded and the number of ReLU neural networks required to achieve an approximation error of $\varepsilon$ is no larger than $2^{b(\varepsilon)}$ with $b(\varepsilon)=\mathcal{O}(\varepsilon^{-m}\log^2(\varepsilon))$. This result improves Lemma IX.4 in Perekrestenko et al. as it shows that the rate at which $b(\varepsilon)$ tends to infinity as $\varepsilon$ tends to zero equals the rectifiability parameter $m$, which can be much smaller than the ambient dimension. We extend this result to countably $m$-rectifiable measures and show that this rate still equals the rectifiability parameter $m$ provided that, among other technical assumptions, the measure decays exponentially on the individual components of the countably $m$-rectifiable support set.