Attention-based Multi-Reference Learning for Image Super-Resolution

TL;DR

This paper introduces an Attention-based Multi-Reference Super-resolution network (AMRSR) that adaptively transfers textures from multiple references to enhance low-resolution images, significantly outperforming existing methods on benchmark datasets.

Contribution

The paper presents a novel hierarchical attention-based sampling method for multi-reference super-resolution, improving texture transfer and image quality over prior approaches.

Findings

Significant performance improvements on benchmark datasets.

Effective handling of diverse reference images.

Ablation studies confirm the importance of multi-reference and attention mechanisms.

Abstract

This paper proposes a novel Attention-based Multi-Reference Super-resolution network (AMRSR) that, given a low-resolution image, learns to adaptively transfer the most similar texture from multiple reference images to the super-resolution output whilst maintaining spatial coherence. The use of multiple reference images together with attention-based sampling is demonstrated to achieve significantly improved performance over state-of-the-art reference super-resolution approaches on multiple benchmark datasets. Reference super-resolution approaches have recently been proposed to overcome the ill-posed problem of image super-resolution by providing additional information from a high-resolution reference image. Multi-reference super-resolution extends this approach by providing a more diverse pool of image features to overcome the inherent information deficit whilst maintaining memory efficiency. A novel hierarchical attention-based sampling approach is introduced to learn the similarity between low-resolution image features and multiple reference images based on a perceptual loss. Ablation demonstrates the contribution of both multi-reference and hierarchical attention-based sampling to overall performance. Perceptual and quantitative ground-truth evaluation demonstrates significant improvement in performance even when the reference images deviate significantly from the target image. The project website can be found at https://marcopesavento.github.io/AMRSR/