Efficient Long-Range Attention Network for Image Super-resolution

TL;DR

The paper introduces ELAN, an efficient long-range attention network for image super-resolution that reduces computational complexity while improving performance by combining shift convolution and group-wise multi-scale self-attention.

Contribution

It proposes a novel ELAN architecture with a new GMSA module and efficient attention blocks, enhancing super-resolution performance with lower complexity.

Findings

ELAN outperforms existing transformer-based SR models in accuracy.

ELAN achieves this with significantly reduced computational complexity.

The model is validated through extensive experiments.

Abstract

Recently, transformer-based methods have demonstrated impressive results in various vision tasks, including image super-resolution (SR), by exploiting the self-attention (SA) for feature extraction. However, the computation of SA in most existing transformer based models is very expensive, while some employed operations may be redundant for the SR task. This limits the range of SA computation and consequently the SR performance. In this work, we propose an efficient long-range attention network (ELAN) for image SR. Specifically, we first employ shift convolution (shift-conv) to effectively extract the image local structural information while maintaining the same level of complexity as 1x1 convolution, then propose a group-wise multi-scale self-attention (GMSA) module, which calculates SA on non-overlapped groups of features using different window sizes to exploit the long-range image dependency. A highly efficient long-range attention block (ELAB) is then built by simply cascading two shift-conv with a GMSA module, which is further accelerated by using a shared attention mechanism. Without bells and whistles, our ELAN follows a fairly simple design by sequentially cascading the ELABs. Extensive experiments demonstrate that ELAN obtains even better results against the transformer-based SR models but with significantly less complexity. The source code can be found at https://github.com/xindongzhang/ELAN.