Improving Super-Resolution Performance using Meta-Attention Layers

TL;DR

This paper introduces meta-attention, a simple, architecture-agnostic mechanism that enhances super-resolution CNNs by utilizing degradation metadata to improve image reconstruction fidelity.

Contribution

The paper presents meta-attention, a novel, easy-to-integrate method that allows any SR network to leverage degradation information without architectural modifications.

Findings

Meta-attention improves PSNR by approximately 0.3 dB on blurred/downsampled images.

The method enhances state-of-the-art SR models' accuracy when degradation metadata is available.

Meta-attention is simple to implement and compatible with various SR architectures.

Abstract

Convolutional Neural Networks (CNNs) have achieved impressive results across many super-resolution (SR) and image restoration tasks. While many such networks can upscale low-resolution (LR) images using just the raw pixel-level information, the ill-posed nature of SR can make it difficult to accurately super-resolve an image which has undergone multiple different degradations. Additional information (metadata) describing the degradation process (such as the blur kernel applied, compression level, etc.) can guide networks to super-resolve LR images with higher fidelity to the original source. Previous attempts at informing SR networks with degradation parameters have indeed been able to improve performance in a number of scenarios. However, due to the fully-convolutional nature of many SR networks, most of these metadata fusion methods either require a complete architectural change, or necessitate the addition of significant extra complexity. Thus, these approaches are difficult to introduce into arbitrary SR networks without considerable design alterations. In this paper, we introduce meta-attention, a simple mechanism which allows any SR CNN to exploit the information available in relevant degradation parameters. The mechanism functions by translating the metadata into a channel attention vector, which in turn selectively modulates the network's feature maps. Incorporating meta-attention into SR networks is straightforward, as it requires no specific type of architecture to function correctly. Extensive testing has shown that meta-attention can consistently improve the pixel-level accuracy of state-of-the-art (SOTA) networks when provided with relevant degradation metadata. For PSNR, the gain on blurred/downsampled (X4) images is of 0.2969 dB (on average) and 0.3320 dB for SOTA general and face SR models, respectively.