S2R: Exploring a Double-Win Transformer-Based Framework for Ideal and Blind Super-Resolution

TL;DR

This paper introduces S2R, a lightweight transformer-based framework with a coarse-to-fine training strategy that excels in both ideal and blind super-resolution tasks, significantly improving performance and convergence speed.

Contribution

The paper proposes a novel double-win framework combining a lightweight transformer model and a two-stage training strategy for superior ideal and blind SR performance.

Findings

Outperforms other SR models in ideal conditions with 578K parameters.

Achieves better results than blind SR models with only 10 gradient updates.

Accelerates transfer learning in real-world SR by 300 times.

Abstract

Nowadays, deep learning based methods have demonstrated impressive performance on ideal super-resolution (SR) datasets, but most of these methods incur dramatically performance drops when directly applied in real-world SR reconstruction tasks with unpredictable blur kernels. To tackle this issue, blind SR methods are proposed to improve the visual results on random blur kernels, which causes unsatisfactory reconstruction effects on ideal low-resolution images similarly. In this paper, we propose a double-win framework for ideal and blind SR task, named S2R, including a light-weight transformer-based SR model (S2R transformer) and a novel coarse-to-fine training strategy, which can achieve excellent visual results on both ideal and random fuzzy conditions. On algorithm level, S2R transformer smartly combines some efficient and light-weight blocks to enhance the representation ability of extracted features with relatively low number of parameters. For training strategy, a coarse-level learning process is firstly performed to improve the generalization of the network with the help of a large-scale external dataset, and then, a fast fine-tune process is developed to transfer the pre-trained model to real-world SR tasks by mining the internal features of the image. Experimental results show that the proposed S2R outperforms other single-image SR models in ideal SR condition with only 578K parameters. Meanwhile, it can achieve better visual results than regular blind SR models in blind fuzzy conditions with only 10 gradient updates, which improve convergence speed by 300 times, significantly accelerating the transfer-learning process in real-world situations.