Signal Recovery from Pooling Representations

TL;DR

This paper derives lower Lipschitz bounds for various pooling operators and demonstrates their invertibility in neural networks through numerical experiments, enabling phase recovery and inverse pooling with empirical validation.

Contribution

It provides new lower Lipschitz bounds for $\, ext{ell}_p$ pooling operators and shows their invertibility in neural networks with practical phase recovery methods.

Findings

Pooling layers can be inverted with phase recovery algorithms.

Lower Lipschitz bounds ensure the regularity of inverse pooling.

Empirical validation on MNIST confirms invertibility and phase recovery.

Abstract

In this work we compute lower Lipschitz bounds of $\ell_p$ pooling operators for $p=1, 2, \infty$ as well as $\ell_p$ pooling operators preceded by half-rectification layers. These give sufficient conditions for the design of invertible neural network layers. Numerical experiments on MNIST and image patches confirm that pooling layers can be inverted with phase recovery algorithms. Moreover, the regularity of the inverse pooling, controlled by the lower Lipschitz constant, is empirically verified with a nearest neighbor regression.