ControlSR: Taming Diffusion Models for Consistent Real-World Image Super Resolution

TL;DR

ControlSR leverages LR information to tame diffusion models, enhancing the consistency and quality of real-world image super-resolution while maintaining fidelity to input images.

Contribution

This work introduces a novel method that effectively utilizes LR information to control diffusion models, improving consistency and quality in real-world image super-resolution.

Findings

Outperforms existing methods on multiple metrics.

Produces more LR-consistent super-resolution images.

Enhances fidelity and visual clarity in SR results.

Abstract

We present ControlSR, a new method that can tame Diffusion Models for consistent real-world image super-resolution (Real-ISR). Previous Real-ISR models mostly focus on how to activate more generative priors of text-to-image diffusion models to make the output high-resolution (HR) images look better. However, since these methods rely too much on the generative priors, the content of the output images is often inconsistent with the input LR ones. To mitigate the above issue, in this work, we tame Diffusion Models by effectively utilizing LR information to impose stronger constraints on the control signals from ControlNet in the latent space. We show that our method can produce higher-quality control signals, which enables the super-resolution results to be more consistent with the LR image and leads to clearer visual results. In addition, we also propose an inference strategy that imposes constraints in the latent space using LR information, allowing for the simultaneous improvement of fidelity and generative ability. Experiments demonstrate that our model can achieve better performance across multiple metrics on several test sets and generate more consistent SR results with LR images than existing methods. Our code is available at https://github.com/HVision-NKU/ControlSR.