PNEN: Pyramid Non-Local Enhanced Networks

TL;DR

This paper introduces Pyramid Non-local Enhanced Networks with a novel non-local module that efficiently captures long-range dependencies across multiple scales, improving low-level image processing tasks like smoothing, denoising, and super-resolution.

Contribution

It proposes a Pyramid Non-local Block that models pixel dependencies across scales, enhancing feature representation while maintaining computational efficiency.

Findings

Achieves state-of-the-art results in edge-preserving image smoothing.

Improves performance in image denoising and super-resolution tasks.

Efficiently exploits multi-scale pairwise dependencies with low memory and computational cost.

Abstract

Existing neural networks proposed for low-level image processing tasks are usually implemented by stacking convolution layers with limited kernel size. Every convolution layer merely involves in context information from a small local neighborhood. More contextual features can be explored as more convolution layers are adopted. However it is difficult and costly to take full advantage of long-range dependencies. We propose a novel non-local module, Pyramid Non-local Block, to build up connection between every pixel and all remain pixels. The proposed module is capable of efficiently exploiting pairwise dependencies between different scales of low-level structures. The target is fulfilled through first learning a query feature map with full resolution and a pyramid of reference feature maps with downscaled resolutions. Then correlations with multi-scale reference features are exploited for enhancing pixel-level feature representation. The calculation procedure is economical considering memory consumption and computational cost. Based on the proposed module, we devise a Pyramid Non-local Enhanced Networks for edge-preserving image smoothing which achieves state-of-the-art performance in imitating three classical image smoothing algorithms. Additionally, the pyramid non-local block can be directly incorporated into convolution neural networks for other image restoration tasks. We integrate it into two existing methods for image denoising and single image super-resolution, achieving consistently improved performance.