Attention in Attention Network for Image Super-Resolution

TL;DR

This paper introduces A$^2$N, an attention in attention network for image super-resolution, which adaptively combines attention and non-attention branches to improve accuracy and efficiency.

Contribution

The paper proposes a novel attention in attention network with a dynamic attention module that adaptively weights branches, enhancing super-resolution performance with fewer parameters.

Findings

A$^2$N outperforms state-of-the-art models of similar size.

Dynamic attention weights adapt to input features, improving SR accuracy.

The model achieves a good balance between performance and complexity.

Abstract

Convolutional neural networks have allowed remarkable advances in single image super-resolution (SISR) over the last decade. Among recent advances in SISR, attention mechanisms are crucial for high-performance SR models. However, the attention mechanism remains unclear on why and how it works in SISR. In this work, we attempt to quantify and visualize attention mechanisms in SISR and show that not all attention modules are equally beneficial. We then propose attention in attention network (A$^2$N) for more efficient and accurate SISR. Specifically, A$^2$N consists of a non-attention branch and a coupling attention branch. A dynamic attention module is proposed to generate weights for these two branches to suppress unwanted attention adjustments dynamically, where the weights change adaptively according to the input features. This allows attention modules to specialize to beneficial examples without otherwise penalties and thus greatly improve the capacity of the attention network with few parameters overhead. Experimental results demonstrate that our final model A$^2$N could achieve superior trade-off performances comparing with state-of-the-art networks of similar sizes. Codes are available at https://github.com/haoyuc/A2N.