Recovery of surfaces and functions in high dimensions: sampling theory and links to neural networks

TL;DR

This paper develops a theoretical framework for recovering signals on high-dimensional unions of smooth surfaces using sampling theory, revealing links to neural networks and low-rank structures.

Contribution

It introduces a novel sampling theory for signals on unions of smooth surfaces and connects low-rank feature properties to neural network-like representations.

Findings

Exponential mapping transforms data to low-dimensional subspaces.

Low-rank features determine the number of measurements needed.

Features provide an efficient neural network-like local function representation.

Abstract

Several imaging algorithms including patch-based image denoising, image time series recovery, and convolutional neural networks can be thought of as methods that exploit the manifold structure of signals. While the empirical performance of these algorithms is impressive, the understanding of recovery of the signals and functions that live on manifold is less understood. In this paper, we focus on the recovery of signals that live on a union of surfaces. In particular, we consider signals living on a union of smooth band-limited surfaces in high dimensions. We show that an exponential mapping transforms the data to a union of low-dimensional subspaces. Using this relation, we introduce a sampling theoretical framework for the recovery of smooth surfaces from few samples and the learning of functions living on smooth surfaces. The low-rank property of the features is used to determine the number of measurements needed to recover the surface. Moreover, the low-rank property of the features also provides an efficient approach, which resembles a neural network, for the local representation of multidimensional functions on the surface.