Anisotropic Pooling for LUT-realizable CNN Image Restoration

TL;DR

This paper proposes anisotropic pooling methods to improve LUT-realizable CNN image restoration, leading to better quality and adaptability over traditional average pooling.

Contribution

It introduces generalized median pooling and learns data-dependent pooling coefficients to enhance LUT-based CNN image restoration.

Findings

Generalized median pooling outperforms average pooling.

Learned orientation-dependent pooling coefficients improve results.

Experimental benchmarks show superior perceptual and numerical quality.

Abstract

Table look-up realization of image restoration CNNs has the potential of achieving competitive image quality while being much faster and resource frugal than the straightforward CNN implementation. The main technical challenge facing the LUT-based CNN algorithm designers is to manage the table size without overly restricting the receptive field. The prevailing strategy is to reuse the table for small pixel patches of different orientations (apparently assuming a degree of isotropy) and then fuse the look-up results. The fusion is currently done by average pooling, which we find being ill suited to anisotropic signal structures. To alleviate the problem, we investigate and discuss anisotropic pooling methods to replace naive averaging for improving the performance of the current LUT-realizable CNN restoration methods. First, we introduce the method of generalized median pooling which leads to measurable gains over average pooling. We then extend this idea by learning data-dependent pooling coefficients for each orientation, so that they can adaptively weigh the contributions of differently oriented pixel patches. Experimental results on various restoration benchmarks show that our anisotropic pooling strategy yields both perceptually and numerically superior results compared to existing LUT-realizable CNN methods.