Time-Aware One Step Diffusion Network for Real-World Image Super-Resolution

TL;DR

This paper introduces TADSR, a time-aware diffusion network for real-world image super-resolution that leverages timestep-conditioned generative priors for improved, controllable one-step super-resolution performance.

Contribution

The paper proposes a novel time-aware VAE encoder and VSD loss to better utilize diffusion priors at different timesteps, enabling effective, controllable one-step super-resolution.

Findings

Achieves state-of-the-art super-resolution results.

Enables controllable trade-offs between fidelity and realism.

Operates efficiently with only a single diffusion step.

Abstract

Diffusion-based real-world image super-resolution (Real-ISR) methods have demonstrated impressive performance.To achieve efficient Real-ISR, many works employ Variational Score Distillation (VSD) to distill pre-trained stable-diffusion (SD) model for one-step SR with a fixed timestep. However, since SD will perform different generative priors at different timesteps, a fixed timestep is difficult for these methods to fully leverage the generative priors in SD, leading to suboptimal performance.To address this, we propose a \textbf{T}ime-\textbf{A}ware one-step \textbf{D}iffusion Network for Real-ISR (\textbf{TADSR}). We first introduce a Time-Aware VAE Encoder, which projects the same image into different latent features based on timesteps.Through joint dynamic variation of timesteps and latent features, the student model can better align with the input pattern distribution of the pre-trained SD, thereby enabling more effective utilization of SD's generative capabilities.To better activate the generative prior of SD at different timesteps, we propose a Time-Aware VSD loss that bridges the timesteps of the student model and those of the teacher model, thereby producing more consistent generative prior guidance conditioned on timesteps. Additionally, though utilizing the generative prior in SD at different timesteps, our method can naturally achieve \textbf{controllable trade-offs between fidelity and realism} by changing the timestep.Experimental results demonstrate that our method achieves both state-of-the-art performance and controllable SR results with only a single step. The source codes are released at https://github.com/zty557/TADSR