Towards a Sampling Theory for Implicit Neural Representations

TL;DR

This paper develops a theoretical framework for understanding the sample complexity of implicit neural representations (INRs) in image recovery tasks, linking non-convex optimization to convex measures and validating with empirical results.

Contribution

It introduces a sampling theory for INRs, relating non-convex training minimizers to convex measures, and provides sample complexity bounds for exact image recovery.

Findings

Identifies sufficient samples for exact recovery with width-1 INRs

Conjectures on sample complexity for wider INRs

Empirically validates recovery probability and super-resolution performance

Abstract

Implicit neural representations (INRs) have emerged as a powerful tool for solving inverse problems in computer vision and computational imaging. INRs represent images as continuous domain functions realized by a neural network taking spatial coordinates as inputs. However, unlike traditional pixel representations, little is known about the sample complexity of estimating images using INRs in the context of linear inverse problems. Towards this end, we study the sampling requirements for recovery of a continuous domain image from its low-pass Fourier coefficients by fitting a single hidden-layer INR with ReLU activation and a Fourier features layer using a generalized form of weight decay regularization. Our key insight is to relate minimizers of this non-convex parameter space optimization problem to minimizers of a convex penalty defined over an infinite-dimensional space of measures. We identify a sufficient number of samples for which an image realized by a width-1 INR is exactly recoverable by solving the INR training problem, and give a conjecture for the general width-$W$ case. To validate our theory, we empirically assess the probability of achieving exact recovery of images realized by low-width single hidden-layer INRs, and illustrate the performance of INR on super-resolution recovery of more realistic continuous domain phantom images.