TASR: Timestep-Aware Diffusion Model for Image Super-Resolution

TL;DR

This paper introduces TASR, a timestep-aware diffusion model for image super-resolution that adaptively combines features from ControlNet and Stable Diffusion, improving image fidelity and detail by leveraging temporal dynamics.

Contribution

The paper proposes a novel timestep-aware diffusion framework that adaptively integrates ControlNet and Stable Diffusion features, with a specialized training strategy for enhanced super-resolution.

Findings

Improved super-resolution quality on benchmark datasets.

Effective early-stage LR information transmission.

Enhanced image detail and fidelity.

Abstract

Diffusion models have recently achieved outstanding results in the field of image super-resolution. These methods typically inject low-resolution (LR) images via ControlNet.In this paper, we first explore the temporal dynamics of information infusion through ControlNet, revealing that the input from LR images predominantly influences the initial stages of the denoising process. Leveraging this insight, we introduce a novel timestep-aware diffusion model that adaptively integrates features from both ControlNet and the pre-trained Stable Diffusion (SD). Our method enhances the transmission of LR information in the early stages of diffusion to guarantee image fidelity and stimulates the generation ability of the SD model itself more in the later stages to enhance the detail of generated images. To train this method, we propose a timestep-aware training strategy that adopts distinct losses at varying timesteps and acts on disparate modules. Experiments on benchmark datasets demonstrate the effectiveness of our method. Code: https://github.com/SleepyLin/TASR