Efficient Image Super-Resolution Using Pixel Attention

TL;DR

This paper introduces a lightweight image super-resolution neural network utilizing a novel pixel attention mechanism that produces 3D attention maps, leading to improved results with fewer parameters.

Contribution

The paper proposes a new pixel attention scheme and two efficient building blocks, achieving comparable performance to larger models with significantly fewer parameters.

Findings

Achieves similar performance to SRResNet and CARN with only 272K parameters

Pixel attention improves super-resolution quality with fewer parameters

Validated effectiveness through ablation studies

Abstract

This work aims at designing a lightweight convolutional neural network for image super resolution (SR). With simplicity bare in mind, we construct a pretty concise and effective network with a newly proposed pixel attention scheme. Pixel attention (PA) is similar as channel attention and spatial attention in formulation. The difference is that PA produces 3D attention maps instead of a 1D attention vector or a 2D map. This attention scheme introduces fewer additional parameters but generates better SR results. On the basis of PA, we propose two building blocks for the main branch and the reconstruction branch, respectively. The first one - SC-PA block has the same structure as the Self-Calibrated convolution but with our PA layer. This block is much more efficient than conventional residual/dense blocks, for its twobranch architecture and attention scheme. While the second one - UPA block combines the nearest-neighbor upsampling, convolution and PA layers. It improves the final reconstruction quality with little parameter cost. Our final model- PAN could achieve similar performance as the lightweight networks - SRResNet and CARN, but with only 272K parameters (17.92% of SRResNet and 17.09% of CARN). The effectiveness of each proposed component is also validated by ablation study. The code is available at https://github.com/zhaohengyuan1/PAN.