RFSR: Improving ISR Diffusion Models via Reward Feedback Learning

TL;DR

This paper introduces RFSR, a reward feedback learning approach that enhances image super-resolution diffusion models by applying a timestep-aware training strategy with regularization, leading to improved perceptual and aesthetic image quality.

Contribution

The paper proposes a novel reward feedback learning method with a timestep-aware strategy and regularization, which can be integrated into existing diffusion-based ISR models to improve image quality.

Findings

Significant improvement in perceptual quality of SR images.

Enhanced aesthetic appeal of generated images.

Method is plug-and-play and compatible with existing models.

Abstract

Generative diffusion models (DM) have been extensively utilized in image super-resolution (ISR). Most of the existing methods adopt the denoising loss from DDPMs for model optimization. We posit that introducing reward feedback learning to finetune the existing models can further improve the quality of the generated images. In this paper, we propose a timestep-aware training strategy with reward feedback learning. Specifically, in the initial denoising stages of ISR diffusion, we apply low-frequency constraints to super-resolution (SR) images to maintain structural stability. In the later denoising stages, we use reward feedback learning to improve the perceptual and aesthetic quality of the SR images. In addition, we incorporate Gram-KL regularization to alleviate stylization caused by reward hacking. Our method can be integrated into any diffusion-based ISR model in a plug-and-play manner. Experiments show that ISR diffusion models, when fine-tuned with our method, significantly improve the perceptual and aesthetic quality of SR images, achieving excellent subjective results. Code: https://github.com/sxpro/RFSR