RRSR:Reciprocal Reference-based Image Super-Resolution with Progressive Feature Alignment and Selection

TL;DR

This paper introduces a reciprocal learning framework for reference-based image super-resolution that enhances existing models by leveraging mutual information between high-quality SR images and their LR counterparts, improving accuracy and robustness.

Contribution

It proposes a reciprocal learning paradigm and a progressive feature alignment and selection module, significantly boosting the performance of existing RefSR models.

Findings

Consistently improves state-of-the-art RefSR models.

Achieves new state-of-the-art results on multiple benchmarks.

Enhances feature transfer accuracy and network robustness.

Abstract

Reference-based image super-resolution (RefSR) is a promising SR branch and has shown great potential in overcoming the limitations of single image super-resolution. While previous state-of-the-art RefSR methods mainly focus on improving the efficacy and robustness of reference feature transfer, it is generally overlooked that a well reconstructed SR image should enable better SR reconstruction for its similar LR images when it is referred to as. Therefore, in this work, we propose a reciprocal learning framework that can appropriately leverage such a fact to reinforce the learning of a RefSR network. Besides, we deliberately design a progressive feature alignment and selection module for further improving the RefSR task. The newly proposed module aligns reference-input images at multi-scale feature spaces and performs reference-aware feature selection in a progressive manner, thus more precise reference features can be transferred into the input features and the network capability is enhanced. Our reciprocal learning paradigm is model-agnostic and it can be applied to arbitrary RefSR models. We empirically show that multiple recent state-of-the-art RefSR models can be consistently improved with our reciprocal learning paradigm. Furthermore, our proposed model together with the reciprocal learning strategy sets new state-of-the-art performances on multiple benchmarks.