Signal Recovery with Non-Expansive Generative Network Priors

TL;DR

This paper proves that signals generated by Gaussian deep networks with contractive layers can be recovered from few measurements, extending previous guarantees to more realistic network architectures.

Contribution

It introduces the Range Restricted Weight Distribution Condition (R2WDC), relaxing the expansivity requirement and enabling recovery guarantees for non-expansive generative networks.

Findings

Gaussian matrices satisfy R2WDC, enabling theoretical guarantees.

Recovery is possible with layer widths proportional to input size.

Extends previous results to non-expansive network architectures.

Abstract

We study compressive sensing with a deep generative network prior. Initial theoretical guarantees for efficient recovery from compressed linear measurements have been developed for signals in the range of a ReLU network with Gaussian weights and logarithmic expansivity: that is when each layer is larger than the previous one by a logarithmic factor. It was later shown that constant expansivity is sufficient for recovery. It has remained open whether the expansivity can be relaxed, allowing for networks with contractive layers (as often the case of real generators). In this work we answer this question, proving that a signal in the range of a Gaussian generative network can be recovered from few linear measurements provided that the width of the layers is proportional to the input layer size (up to log factors). This condition allows the generative network to have contractive layers. Our result is based on showing that Gaussian matrices satisfy a matrix concentration inequality which we term Range Restricted Weight Distribution Condition (R2WDC) and that weakens the Weight Distribution Condition (WDC) upon which previous theoretical guarantees were based. The WDC has also been used to analyze other signal recovery problems with generative network priors. By replacing the WDC with the R2WDC, we are able to extend previous results for signal recovery with expansive generative network priors to non-expansive ones. We discuss these extensions for phase retrieval, denoising, and spiked matrix recovery.